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TITLE 

METHOD OF DETERMINING THE INTERIOR 
POINTS OF AN OBJECT IN A BACKGROUND 
5 This application is a continuation-in-part of 

application Serial No. 07/767,339, filed September 27, 
1991. 

MICROFICHE APPENDIX 
Attached hereto are microfiche containing 8 6 

10 frames of modules which can be employed in the 

described embodiments and in other embodiments. These 
microfiche are included as a portion of the disclosure 
of this patent document. 

COPYRIGHT NOTICE 

15 A portion of the disclosure of this patent 

document contains material which is subject to 
copyright protection. The copyright owner has no 
objection to the reproduction by anyone of the patent 
document or the patent disclosure, as it appears in the 

20 Patent and Trademark Office patent file or records, but 
otherwise reserves all copyright rights whatsoever. 

PACKSPOUNP OT THE INVENTION 
i - Field of the, Invention 

The present inyention relates to an image analysis 

25 method for determining the interior points of an object 
in a background/ The present invention also .relates to 
a method of identifying at least one valid object 
having at least one-predetermined attribute value in a 
background. By determining the interior points of the 

30 object, one may "blacken ; out " or effectively remove 
previously identified objects from an imago of the 
object and the background, thus creating a remainder 
image for further searching. 
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2. no^T-ipti Ton of the Related Art 

The ability. ( t.o acce.s? interior points of a region 
when its contour is given is a classic problem in image 
processing. Previous splutions to this problem have 
been described by L . W . Chang and K. L. Leu in an 
article entitled "A Fast Algorithm for the Restoration 
of. Images Based .on Chain Codes Description and Its 
Applications",, Computer Vision, Graphics, and Image 
Processing. 50, 29.6-307 ; (1990) . One algorithm 
described, connectivity filling, requires that every 
pixel in the object, be interrogated. Since the number 
of pixels within an object is greater than the number 
of pixels which comprise the perimeter of the object, 
such a technique is relatively slow. Another method 
described is parity checking which is based on the fact 
that the intersection of a straight line with the 
contour- regions of an object delineates an even number 
of points on the perimeter of the object. Designation 
of- the interior points is determined by an arbitrary 
filling rule where, all points between each odd point 
and the next even, .point on the contour in each line Of 
the image are designated as interior points. Such a 
parity checking algorithm wi.ll not work with objects of 

complex shape. .: , 

■ In the same article cited above, Chang and Leu 
propose, an enhanced parity checking algorithm which 
relies., on characterizing the perimeter points into one 
of four categories -based on. direction code values. 
Characterizing .the points leads to the creation of a 
new buffer of perimeter points wherein some points have 
been duplicated and others eliminated. By applying the 
parity checking algorithm to the newly created buffer, 
.interior points of objects with complex shapes may be 
identified. However, the enhanced algorithm presented 
by Chang and Leu is an indirect method for solving the 
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problem' of interior point determination, requiring an 
excessive number of "steps . 

Japanese Patent Publication Number 62-60069 to 
Hitachi Electronic Engineering Co. Ltd., published 
5 12/14/87, discloses * a colony screening device which 
distinguishes colonies based on color . Colonies are 
identified by pattern' recognition using a predetermined 
threshold and a small "number of points' in the center of 
a colony are arbitrarily selected'. : This publication 
10 "does hot disclose a method f or ' determininig -all the 
interior points of an object. Limiting the area of 
determination introduces possibilities of error into 
any attempt at 6b jedti identification and makes a search 
of the remainder' portion of ' the image less effective. 
15 U.S. Patent No . 453, 266 -t6' Bacus discloses a 

method for measuring the optical- "density of a cell 
using the sum of gray level values ' within the closed 
boundaries of the cell. Bacus does hot contemplate 
'determining all of the interior points of an object of 
20 an unknown or arbitrary shape using an automatic 
thresholding techniques. 

None; of these references discloses a method for 
(determining all the interior points of the object 
identified. Furthermore, none of these references 
25 discloses a method of identifying at least one valid 
object having- at least one valid object predetermined 
attribute value in a background U3 : ing automatically 
calculated thresholds by determining the interior 
points of the object. ' Moreover, none of these 
30 references discloses a method for effectively removing 
previously identified object from an image, thereby 
creating a remainder image for further searching. 

SUtiMARY OF THS INVENTION 
Accordingly, the present invention solves the 
35 problems of the prior art by providing a method for 
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determining the- interior .points .of the object 

identified. . . .-<r -■:.-■!'■■ " : 

The present invention also s.olyes the problems of 
the prior art .by providing a. method of identifying 
5 objects having at least one predetermined attribute 
value in a background using automatically calculated 
thresholds by determining the interior points of the 
objects. identified. ■ Particularly ,. the method of the 
present; 'invention- -is rable to identify objects in a 

10 varying, background. 

- ■ The present invention further solves the problems 
of the>.prior art by providing a method for effectively 
: removing previously - identified object, .from an image, 
thereby creating- a remainder image for further 

'15 searching. ■■ ' 

The method- of the present invention also solves 

. the problems of the prior. arfc by using filling rules, 
which are specif ic to- each perimeter point of an 
object, to provide a direct method for determining the 
20 interior points of the object.. 

To achieve the foregoing solutions, and in 
• accordance the purposes of a first embodiment of the 
invention as embodied and broadly described herein, 
there is provided a method for determining the interior 
- 25 • points: of an object in a background. The method 
comprises the steps of generating an image of the 
object 'and the 'background, searching the image for the 
object and" determining the perimeter points of the 
object. The -method also comprises the step of creating 
30 a perimeter, buffer... The perimeter buffer comprises the 
: perimeter points, . where each- perimeter point has a row- 
position coordinate value, a column -posit ion coordinate 
value and a direction code vector. The direction code 
vector comprises a first and a second direction code 
35 value for each perimeter point. The first direction 
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code value 'describes the relationship of each 
respective perimeter point to the next perimeter point 
and the second direction code: value describes the 
relationship . cf each respective, perimeter point to the 
5 previous perimeter -point , The' method also comprises 
the step of sorting the perimeter points in the 
perimeter buffer in a predetermined order to create a 
sorted perimeter buffer;, . and assigning to each 
perimeter point in thevsorted perimeter buffer a status 
10 variable having one of : a first and a -second value, 

where the value of - the status , variable is determined by 
the first and second: direction code values. The method 
also- comprises the step of - designating as the interior 
points, of the object each point on- the same line 
15 segment as the perimeter point. being examined and lying 
between the examined perimeter point and the next 
perimeter point in: the- sorted perimeter buffer, where 
the examined perimeter point is assigned the firsts- 
value of the "status variable. 
20 Further in accordance with the purposes of a 

second-, embodiment of the, invention as embodied and 
broadly described herein, there is provided a method of 
identifying at least: one valid object having at least 
one predetermined attribute value in a background. The 
25 method comprises the steps of generating an image of 

the object and the -background, searching the image for 
at least one candidate- object using at, least one 
automatically- calculated threshold- and determining the 
perimeter- points. of the candidate object. The method 
30 also comprises the step. of creating a perimeter buffer. 
The perimeter buffer comprises the perimeter points, 
where each perimeter point has a row^position 
coordinate value, a column-position coordinate value 
and a direction code vector. The direction code vector 
35 comprises a first and a second direction code value for 
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each perimeter-point. ■ The first direction code value 
describes the relationship of each, respective perimeter 
point to the. next perimeter point . and the second 
direction code value describes the relationship of each 
respective perimeter point to the previous perimeter 
point . The method also comprises the .step of sorting 
the perimeter points in the, perimeter buffer in a 
predetermined order to create a sorted, perimeter 
.buffer, and assigning to each perimeter point in the 
sorted perimeter buffer a status variable having one of 
a first and a second value., . where the. value of the 
status variable is determined by the first and second 
direction code values,. . The method also comprises the 
step of designating as the. interior points of the 
object each, point on the same line segment as the 
perimeter point being examined and lying between the 
examined perimeter point and the next perimeter point 
in the sorted. perimeter buffer, where the examined 
perimeter., point is assigned the first value of the 
status variable. The . method also comprises the steps 
characterizing the candidate object by the candidate 1 
object attribute value and validating the candidate 
object having the valid, object predetermined attribute 
value to identify the valid object. 

,In . order, to remove, a previously identified object 
from an image, there is provided a method for creating 
a remainder image for further searching . The method 
comprises the steps of the second embodiment, and in 
addition, comprises the steps qf selecting a 
predetermined gray level value ,and scanning the image 
to determine all points having the predetermined gray 
level value, re-assigning the gray level value of each 
point having the predetermined gray level value a new 
predetermined gray level value and assigning the 
predetermined gray level value to all interior points 
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of all valid - objects - ' The method also comprises the 
steps of generating a gray level histogram of a 
remainder image, 'where the remainder" image comprises 
all the points having "a gray level value other than the 
5 predetermined gray level value, selecting N global 

entropic threshold level's, subdividing the gray level 
histogram into N + 1 sub-histograms 'using each of the 
entropically select e& { threshold' gray level values, 
searching portions of ; the image corresponding to each 
10 sub-histogram using each' global, entropically selected 
threshold 'gray level value for at- least one new 
candidate object, wherein the new candidate object has 
at^ least one candidaitie object attribute value and 
1 validating the new' candidate objects having the valid 
15 ' object predetermined attribute- values for each of the 
sub-histograms to identify the valid* object. 

BRIEF DESCRIPTION OF- -THE DRAWINGS 
The ■ accompanying drawings, which are incorporated 
in and constitute a part of the specification, 
20 illustrate the presently preferred embodiments of the 
invention and f together with the general description 
given above and the detailed description of the 
preferred embodiments given below, serve to explaining 
the principles of the invention. 
25 Fig. l"is a -block diagram showing the steps of the 

overall method according to a first embodiment of the 
present invention. 1 - 

Fig. 2A is a schematic view , with rows and columns 
designated, 1 which represent the row and column position 
30 coordinate values', of a simple object traced in a 
clockwise direction. ' 

Fig. 2B is a chart 'showing the row-position 
coordinate value, the column-position coordinate value 
and the direction code values for the thirteen points 
35 illustrated in Fig. 2A. 
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Fig. 3 is a flow chart showing the steps of a 
module, GET PERIM, for determining the coordinates of 
each, perimeter point of an object. 

Fig. 4 is a schematic view' illustrating the 
.5 direction code values for an object which is traced in 
the clockwise direction . 

Fig.. 5 is a^chematic view illustrating the 
direction code values for an object which is traced in 
the counterclockwise direction. 
10 Figs. 6A and 6B are a flow chart showing the steps 

of a module, LOAD BUFFER, which creates a perimeter 
buffer comprising the perimeter points of an object. 

£ig. 7 is a flow chart showing the steps of part 
of ; a module,, READ. COLOR, which assigns a status 
15 , . ; variable to each perimeter point. 

Fig. 8 is a FILL look-up table for objects traced 
in the clockwise direction. 

Fig. 9 is a FILL look-up table for objects traced 
in the counterclockwise, direction. 
20 Figs, 1 OA and 10B are a flow chart showing the 

steps of part of the module READ COLOR for the ? 
..implementation of the first embodiment of the present 
invention where the perimeter points in the perimeter 
buffer are first sorted by. row, then by column. 
25 Fig. 11A is a schematic diagram showing a Red- 

Green-Blue color space. 

Fig. 11B is a schematic diagram showing a Red- 
Green-Blue hue circle. 

..Fig. 12 is a flow chart showing the steps of a 
30. module, HISTOGRAM, which is used to generate a gray 
level histogram of an image. 

Fig. 13 is a flow chart showing the steps of a 
module,. ENTROPY, which is used to entropically select a 
threshold gray, level such that the entropy function of 
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the histogram is maximized according to a specific 
application of the first embodiment . 

Fig. 14 is a gray level histogram of an image of a 
single, simple object in a varying background. 
5 Fig. 15 is flow chart showing the' steps of a 

module, SEARCH IMAGE, which is used to search an image 
for at least one candidate object. 

Fig. 16 is a flow chart illustrating the steps of 
a module, FIND OBJECT, which is " also " used to search the 
10 image . 

Fig. 17 is a block diagram showing the overall 
method of a second embodiment of the present invention. 

Fig. 18 is a flow chart showing the steps of a 
module, COL FILT, which is used to filter objects. 
15 Figs.' 19A -19B are a flow chart showing the steps 

of a module, TRACE OBJECT, which is used to trace 
objects. 

Fig. 20 is a flow chart illustrating the steps of 
a module, CHK GRAY, which is used "to detect whether a 
20 candidate object is relatively lighter or darker than 
the background. 

Figs. 21A - 21C are original',* upper and lower gray 
level histograms, respectively, of an image for a 
first, specific implemeritiation of the second embodiment 
25 of the present invention . 

Fig. 22 is a 'flow' chart showing the steps of a 
module, ANALYZE, which is used to recursively search 
the image for candidate objects according to the first, 
specific implementation of the second embodiment. 
30 Figs^ 23A - 23D are a flow chart showing the steps 

of a module, CHK LIST, which is used to resolve 
redundancies in inhomogeneous objects . 

Fig. 24 is a flow chart showing the steps of a 
module, SET STAT, which is used with the module CHK 
35 LIST as shown in Figs. 22A - 22D. 
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Figs. 25A.-..25B are a flow chart showing the steps 
of a. module, CHK XIST r which is used to resolve 
redundancies in homogeneous objects. 

, . Fig. 2 6... is a flow chart showing the steps of a 
module, SET STAT, which is used with the module CHK 
LIST, as shown, in Figs. _25A - 25B. 

Figs. 27A - 2 7B are a flow chart showing the steps 
of a module, FINAL. CHK, which is ; used to perform a 
final check to resolve redundancies in inhomogeneous 
and homogeneous objects. 

Fig. .28 is. a . flow chart showing the steps of a 
module,, INT STAT, which is used with the module FINAL 
CHK as shown in , Figs. 27A - 27B. 

Figs. ,.29A - . 29B are a block diagram showing the 
steps of the overall method of a second, specific 
inpl-emer.tat.ion of the second embodiment of the present 
invention . . 

Fig. 30 is a gray level histogram which has been 
divided into N + 1 sub-histograms in accordance with 
the second, specific application of the second 
embodiment 

Fig. 31 t .is a schematic diagram showing the 
-components of a. system used with the first and second 
embodiments ,o.f the present invention. 

?rTr.! LED DESCRIPTION OF THE PRE FERRED EMBODIMENTS 

Reference will now be made in detail to the 
present, preferred ^embodiment s . of the invention as 
illustrated in the accompanying drawings. 

In accordance with r a first, embodiment of the 
present invention, there is provided a method for 
determining the .interior points of an object in a 
background. .-Fig- 1 is a block diagram showing the 
overall method .of the present invention. The method 
comprises the step, of generating an i^age of the object 
and the background as shown in block A of Fig. 1. The 
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hardware used 1 to* implement the method of the present 
invention must" fifst "be initialized when the image is 
generated. Th^ image of the object and the background 
may be generated by a camera. Although a CCD camera is 
generally used with the present invention, any type of 
camera "may be used without departing from the general 
principles of the present invention. The image is then 
digitized and stored by a frame grabber or a video 
digitizer. 

The method of the present' invention also comprises 
the step of searching the image for the object as shown 
in block B of Fig. 1. The searching step may comprise 
searching the entire object- In instances where it is 
known' that an identifying feature of an object in 
question lies in a particular portion of an object, the 
searching 1 step may * comprise searching only that portion 
of the object- The searching step may be done in any 
manner/ for instance, using an automatically calculated 
threshold or a user selected threshold. 

"The method of the present invention further 
includes the step of determining the perimeter points 
of the 'object, which is done by tracing the object. 
This step is showii in block C of Fig. "1. The 
coordinates of eaLch perimeter point are determined by a 
module GET PERIM ? . frie -steps of GET PERIM are 
illustrated* in : Fig. T 3 . " The"- basic principles of the GET 
PERIM module are' sim'iiVr to those described in "Digital 
Image Processing" "by Ka'fael'C. Gonzalez and Paul Wintz, 
Second Ed. , Addisbn-We'sley Publishing Company, Reading, 
Massachusetts (1987). In one instance, the step of 
determining the perimeter points of the object 
comprises tracing the object sequentially in a 
clockwise direction- In another instance, the step of 
determining the perimeter points of the object 
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comprises tracing the .object sequentially in a 
counterclockwise direction ... 

The method of the present invention also comprises 
the step of creating a perimeter buffer. This step is 
shown generally by block D of Fig. 1 and is performed 
by a module,.. LOAD BUFFER, as shown specifically in Fig. 
6 as described below. The perimeter buffer comprises 
the perimeter points as determined by the module GET 

PERIM.- . - < . - , . 

Fig. 2A shows a. schematic, for a simple object 
traced .in a clockwise direction. For. purposes of 
illustration, thirteen perimeter, ppints are shown. 
Each perimeter point has a row position coordinate 
value,, a column position coordinate value and a 
direction .code vector- The direction code vector 
comprises a first and a second direction code value for 
each perimeter point. The first direction code value 
.describes the relationship of each respective perimeter 
point to the next perimeter point as determined by the 
GET PERIM module, and the second direction code value 
describes the relationship of each, respective perimeter 
point to the previous perimeter point, as determined by 
the GET PERIM module. The definition of "next or 
previous point", as .discussed below, depends on the 
direction in which. the object is traced. The i ' th 
member of the perimeter buffer is denoted by the 
direction code descriptor (x^, y L , vector^ , where x L is 
the. row. position coordinate value, yi is the column 
position coordinate .value and vector^ is the direction 
code vector .which contains the following information: 

(i) . .nexti is the direction code value to 
the next perimeter point; and 

(ii) previ is the direction code value from 
the previous point to the current point. 
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Fig. 2B is a ~ chart* showing* the row-position 
coordinate value, '* the column-position coordinate value, 
and the first and second direction code values for the 
thirteen p>oints illustrated in Fig. 2A. For example, 
5 point 13 is denoted" as having a . row-position coordinate 
value cf 2, a column-position coordinate value of 1, a 
first direction code value of 1 and a second direction 
code value of 7. ' " •* 

In the present invention, the' -direction code 

10 values can range from 0* : - 7, depending on the direction 
of one perimeter point to an* adjacent perimeter point 
(either "the next perimeter point or the previous 
perimeter' point) . The:' direction code values for the 
present invention are shown in Figs.' 4 and 5. Fig. 5 

15 direction code values are derived by rotating the 

values illustrated In Fig. 4 by 90 ° . ' The application 
of the set of direction code- values as* illustrated in 
Fig. 4 as opposed to the direction code values as 
illustrated in Fig* 5 is dependent upon the method of 

20 sorting the perimeter buffer to be discussed below. As 
can be. seen from Fig. 4 r if next^ is equal to zero, the 
next perimeter point would be vertically above the 
current perimeter point and in the same column. It is 
Important to realize that the value of: a direction code 

25 from one perimeter point to the next 'depends on the 

sense in which the object Is traced- by the module GET 
PERIM. 1 If the* object is traced in the clockwise 
direction, the direction codes, associated with the 
perimeter points 'are different than if the object is 

30 traced in' the counterclockwise direction. 

The steps of the module GET PERIM are shown in 
Fig. 3. As illustrated in Fig. 3, block A sets the 
current* point to the' first point in the object. 
Decision diamond B then asks whether the next perimeter 

35 point has been found. If not, then the traced object 
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is invalid, ^and the module is . done: , as shown in block C. 
If the next ; perimeter point has . been found, then the 
.row and column position coordinate, values are stored in 
the perimeter buffer, .as illustrated , in block D of Fig. 
3. The current point is set to the next perimeter 
point as shown : in block E of Fig,. 3. .Decision diamond 
F then asks whether the current point is at the first 
-perimeter .point ,.-.If. not,, then,. the loop through 
decision diamond B,- ..blocks C - E and decision diamond F 
is repeated until the . current point is at the first 
perimeter point.. ,-If the answer to decision diamond F 
is yes, the module GET PERIM is done, ; 

The -steps of the- module LOAD BUFFER, which creates 
a perimeter buffer,, /are illustrated ..in Figs. 6A and 6B. 
In the first- step as ^hown in block A, the current 
•point is set .to • t ; he first point in the perimeter 
buffer. Decision diamond B - then asks whether the 
current point is beyond the last point in the buffer. 
If it is, then :the LOAD BUFFER module is done as 
illustrated in block G of Fig. 6A. If the current 
point is not beyond the last point in the buffer, then 
decision diamond D asks if : . the current point is the 
first point in. the buffer- If.it is, then block E sets 
the previous point as -the last -point in the buffer. If 
the current point is. not- the first, point in the buffer, 
then decision diamond Flasks if the current point is 
the last point in the buffer. If*. it. is, then block G 
sets' the next point • as the first point in the buffer- 
After the previous? -point, is. set as the last point in 
the perimeter buffer,, or the next point is set as the 
first point in the^perimeter buffer, the difference in 
the row coordinate value between the next point and the 
current point is calculated and stored in the buffer as 
shown in block H in Fig. 6A. This difference is 
designated as A ROWne^. The difference in the column 
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coordinate value between the next point and the current 
point is - then calculated and stored in the buffer as 
shown in 'block I in Fig. 6A. This -difference is 
designated as A COL NEXT ; The difference in the. row 
5- coordinate value between the current point and the 
previous point is then calculated and stored in the 
buffer as ' shown in blo'ek J in Fig; 6B. This difference 
is designated as A ROW PREV ; . Next ; .: the- difference in 
the column coordinate value between, the current point 

10 arid th£ previous point is calculated and stored in the 
buffer as shown in block K. This difference is 
designated as A COL pREV . i. .-The current point is then 
moved to the next perimeter point in the buffer as 
shown in block L,- and -the module returns to decision 

15 diamond B of Fig.- 1 6A» The loop .through B and D - L 
continues until -the current point is beyond the last 
point in the buffer/- at which point the. LOAD BUFFER 
module' is done as -shown in block ;Cv 

"The method of the present invention also comprises 

20 the step of sorting the perimeter points in the 

perimeter buffer in a predetermined order to create a 
sorted perimeter buffer. This step is. :shown in block E 
in Fig. a: This sorting step is. performed for tracing 
the object in either the clockwise. or the counter- 

25 clockwise direction; This sorting- step is performed by 
the standard "Shell Sort", technique described by Robert 
Sedgewick in His t&xtbook: Algorithms in C, Addison 
1 Wesley", 1990. ^ Tt should^ be noted, that the "vector^' 
information associated with each perimeter point moves 

30 with r the "point during the sort . . 

In a first implementation -of. the first embodiment 
of t'he present invention, the object is traced in the 
clockwise direction, and the sorting step comprises the 
sub-step of sorting the perimeter points in the 

35 perimeter buffer by row in a predetermined order to 
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create a row-sorted perimeter .buffer comprising at 
least one row-sorted perimeter point. The sub-step of 
sorting , the . perimeter .points ,by -row comprises sorting 
the perimeter points in ascending order such that the 
5 points with the ,lowest ; row-position . coordinate values 
are placed- at the start (meaning the first element of 

- the sequence. of £he buffer) of the buffer. 
Alternatively, the sub-step of sorting the perimeter 
points. -by r.ow comprises , sorting the perimeter points in 

.10- descending order-such that the points with the highest 
row-position coordinate values are placed at the start 

- of the buffer. - 

, - In. this implementation, of the first embodiment, 

the sorting step; also comprises , the additional sub-step 

15 of sorting the -perimeter points, in the row-sorted 

perimeter buffer, by column in a further predetermined 
order to create a row-column sorted perimeter buffer 
comprising at least one row-column sorted perimeter 
point. .The sub-step, of sorting the perimeter points by 

20 column; comprises .sorting the perimeter points in 

ascending order. In the case where the buffer has been 
sorted ;by row in ascending order and subsequently by 
column; the point with the lowest , column-position 
coordinate value and. the lowest row-position coordinate 
-25 value is placed at the start of the buffer. In the 
case where the: buffer has. been sorted by row in 
descending ..order and, subsequently by column, the point 
with the lowest -column-position coordinate value and 
the .highest row-position coordinate value is placed at 

30 the start of the buffer. * . t . ; 

In a. second implementation of the first 
embodiment, the object is traced in, the clockwise 
direction, and the sorting step comprises the sub-step 
of sorting: the t perimeter points in the perimeter buffer 

35 by column in a predetermined order to create a column- 
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sorted perimeter buffer 'comprising* at least one column- 
sorted perimeter point. In this implementation, the 
sorting step comprises the sub-step of sorting the 
perimeter points in the. column-sorted perimeter buffer 
5' by row ih a predetermined order to create a column-row 
sorted perimeter buffer comprising at least one column- 
row sorted perimeter point. ^The sub-step of sorting 
the perimeter points by colttmtt comprises sorting the 
' perimeter points in ascending order - such that the 
10 points 7 with the lowest- column-position coordinate 
values are placed at 1 the start 1 .of the buffer. 
Alternatively, the sub-step v of sorting the perimeter 
points by column comprises sorting the perimeter points 
• in descending order' 'such' that the points with the 
15 highest- coiumii-posit-ioh coordinate values are placed at 
the start of the buffer. 

In this second- implementation , : the sorting step 
also comprises the additional sub-rstep of sorting;- the 
perimeter points in the" column-sorted buffer by row in 
20 a further predetermined order to create a column-row 

- sorted 'perimeter buff er comprising at least one column- 
row sorted perimeter point. The sub-step of sorting 
- ■ ' ' the perimeter points by row comprises sorting the- 

column-sorted perimeter points -in descending order. In 
25 the- case where the buffer has been sorted by column in 
ascending order and subsequently by row, the point with 
the highest -row-position coordinate and lowest column- 
position coordinate 1 : is placed at the start of the 
•buffer. ; In the- case where the buffer has been sorted 
30 by column in descending order and then by row, the 

point with the highest row coordinate value and highest 
column- coordinate value is placed-at the start of the 
*" buffer. 

In a third implementation of the first embodiment 
35 of the present invention, the object is traced in the 
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counterclockwise direction,, and th^ sorting step 
comprises the sub-step .of . sorting the perimeter points 
in the .perimeter .buffer by row in a predetermined order 
to create a row-sorted perimeter buffer comprising at 
least one row-sorted, perimeter point. The sub-step of 
sorting the perimeter points by row comprises sorting 
the perimeter points in ascending order such that the 
points ,with the lowest row-position coordinate values 
are .placed at the start of the buffer. Alternatively, 
the sub-step ,_of sorting the perimeter points by row 
comprises ..sorting, the perimeter points in descending 
order such that /the points with the highest row- 
position coordinate values are placed at the start of 
the buffer., . . 

_.In this third implementation, the sorting step 
also comprises the additional sub-step of sorting the 
perimeter points in the row-sorted perimeter buffer by 
column in a. further predetermined order to create a 
row-column sorted perimeter buffer comprising at least 
one row-column sorted perimeter point. The sub-step of 
sorting the row-sorted perimeter points by column * 
comprises sorting the perimeter points in descending 
order.. In the case where the buffer has been sorted 
firs.t by row .in .ascending order and subsequently by 
column, the point with. the highest column-position 
coordinate .and lowest row-position coordinate is placed 
at the start of . the buff er . In the case where the 
buffer has been, sorted first by row in descending order 
and then, by column, the point . with the highest column- 
position coordinate, and highest row-position coordinate 
is placed at the start of the buffer. 

In a fourth implementation of the first embodiment 
of the present invention, the object is traced in the 
counterclockwise direction, and the sorting step 
comprises the sub-step of sorting the perimeter points 
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in the perimeter buffer by column in a predetermined 
order to create 'a column-sorted perimeter buffer 
comprising 'at lea3t one column-sorted- perimeter point. 
The sub-step of sorting the perimeter points by column 
5 comprises sorting the perimeter points* in ascending 
order such that ' the p'birit sf' with the lowest column- 
position coordinate values "aire placed at the start of 
the buffer. Alternatively ', the 'sub-step of sorting the 
perimeter points by colukn comprises 'sorting the 
10 perimeter points *iri" descending 'order such that the 
points with the highest ' column-po'sitioii coordinate 
values are placed at the start 'of the buffer. 

In this' fourth implementation, the sorting step 
also comprises the additional sub-step of the sorting 
15 of the perimeter points in' a dolumn-sorted perimeter 
buffer by row* in a further predetermined order to 
create a column-row sorted perimeter buffer comprising 
at least one column-row sorted perimeter point. The 
sub-step of sorting the' perimeter points by row in the 
20 column-sorted buffer comprises sorting 1 the perimeter 
points, in ascending order. In the case where the 
buffer has been sorted first by column in ascending 
order and subsequently by row, the point with the 
lowest row-position coordinate value and lowest column- 
25 position coordinate value is placed at the start of the 
buffer - , 'In the "case where the buffer has been sorted 
first by column in descending order and subsequently by 
row, the point with the' lowest row-position coordinate 
value and highest- column' position coordinate value is 
30 placed' at the start of the buffer. 

The method of the present invention also comprises 
the step of assigning to each perimeter point a status 
variable having one of a first and a second value. 
This step is shown generally in block F of Fig. 1 and 
35 is performed in a module, READ COLOR, as shown in the 



BNSDOCID: <WO 94164Q2A1_I_> 



SUBSTITUTE SHEET 




WO 94/16402 PCT/US93/02736 

• 20 

flow chart of Fig.. 7., Jhe first value of the status 
variable. is "FILL" , and the ; second value of the status 
.variable is "NO FILL". The value of. the status 
variable is determined^ by the first and second 
5 direction, code .values . nexti and prev A . Direction code 
values are determined in accordance with Figs. 4 and 5, 
depending ..on . the method of sorting the perimeter buffer 
which is .employed. _ If the points in the perimeter 
. buffer .are .sorted first by row and then by column, such 
10, as in the. ..first and third implementations of the first 
embodiment., .the direction code values in Fig. 4 are 
.employed. If . the. r points in the perimeter buffer are 
t , . sorted first by column and then by row, such as in the 
: second and fourth implementations of the first 
15- embodiment, then the direction code ; values illustrated 
; -in Fig. 5 are used... The rules for mapping next ± and 

: .. prev i ,to FILL or NO. FILL , are summarized for objects 
- ( traced in ; a clockwise direction in a FILL look-up table 

{ LUT) .as shown in Fig. 8. . The rules for mapping next ± 
• 20 and prev ±; to FILL or NO FILL are summarized . for objects 
traced, in a counterclockwise direction in a FILL look-- 
up table (LUT) as shown in Fig. 9.. In the look-up 
- : , tables of Figs. ,8 and 9, F stands for FILL and NF 

stands for NO FILL. The look-up tables of Figs. 8 and 
25 r 9 are empirically, derived, based on the local concavity 
or convexity of, the object. 

As, shown in , block , A of Fig. 7, the current point 
is set f to the first point in the perimeter buffer. 
Decision diamond B.then. asks if the current point is 
30 beyond the last point in the buffer. If so, then the 

module is done as. shown in block C. Using the look-up- 
table as shown in Fig. 8, the value of the status 
variable, is. assigned, based on the first and second 
direction code values, next ± and prev ± as shown in 
35 block D of Fig. 7. The current point is then moved to 
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the next point' in the perimeter ' buff er as shown in 
"biock^E of Fig. 7."" The loop through decision diamond B 
and blocks D and E is repeated until the current point 
is beyond the last point in the' buff er, at which point 
5 the part of READ" COLOR as shown in Fig. 7 is done as 
shown in block C. 

The method of "the first; embodiment of the present 
invention further comprises' the step of designating as 
the interior points of 'the object each point on the 

10 same line 'segment as ' the perimeter ^poiht being examined 
and lying between the examined perimeter point and the 
next perimeter point -in" the sorted perimeter buffer, 
where the examined perimeter point is assigned the 
first value, FILL, of the' "status variable. This step 

i5 is shown generally in "block G o'f 1 Fig ' 1 and ( is 

performed in the module' READ COLOR; The flow chart for 
the 'first implementation' bf the '-first embodiment of the 
present invention is illustrated in* "Figs. 10A and 10B, 
it being understood that analogous flow charts exist 

20 for the second, third and fourth implementations , In 
the first and third implementations; where the 
perimeter points in ' the perimeter buffer are first 
sorted by row, then by column, the designating step 
comprises the sub-step of designating as the interior 

25 points of the object " each point on the same row as the 
perimeter point being * examined and lying between the 
examined perimeter point and 1 the next perimeter point 
in the row-column sorted perimeter buffer. In the 
second and fourth 'implementations , where the perimeter 

30' points in the perimeter buffer are first sorted by 

column, then by row, ~the designating step comprises the 
sub-step of designating as the interior point of the 
object each' point on the same column as the perimeter 
point being examined and lying between the examined 
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perimeter- point and the- next; perimeter point in the 
column-row sorted perimeter buffer. 

The method of the first embodiment of the present 
invention further comprises . the step of checking for 
5 redundant perimeter points. This step is performed in 
the module,: READ COLOR, as illustrated . in Figs. 10A and 
10B by..: decision diamond E and block F. A redundant 
perimeter point is- one : which has been accessed or 
traversed more than once .during the sequential tracing 
' 10 f of the object. For example, this could occur at cross- 
over points , which is a point which has been crossed 
over more than once during the tracing step. When this 

- 5 redundant interior- point; ^checking is performed, the 

• ,( <s-tep of designating- all points lying between each 

15 ■ r - redundant perimeteri.'point .and..; the next perimeter point 
on the same Aline segment as the interior points of the 
-object occurs, only .when all instances of each redundant 
perimeter point are /assigned the : first value of the 
status variable (FILL). 

20 - - As shown in block A of Fig. 10A, the first step of 
READ COLOR which is run for the first implementation of 
the first * embodiment is to set the. . current point to the 
first point of the perimeter buffer. Decision diamond 
3 then asks whether the current point is beyond the 
- 25 last point of - the perimeter buffer. If so, the module 
is done as shown in block C. , If the< ; current point is 
not beyond the last point, then, the current point is 
labeled as ; an interior .point as shown in block D. Then 
decision diamond E asks whether ■ the current point is 

30 the same point as the. next point in-- the buffer. If it 
is, then the current point, is labeled as redundant as 
shown in block F . Then decision diamond H asks if the 
status variable of all instances of the current point 
is equal to FILL. If the answer is yes, then READ 

35" COLOR moves to the next point on the same row as the 
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current point as shown in block J.* If the answer is 
no, the current point is set to the next point in the 
buffer as shown in block I and the module returns to 
decision diamond • Returning to decision diamond E, 
if the current point is : - not the- same point as the next 
point in the buffer, then decision, diamond G asks if 
the status value of the current point, is equal to FILL. 
If : it' is not, the "module returns to decision diamond B. 
The loop through - B and - ; D - I ■■ continues - until the 
current point is -beyond the last pointy at "which time 
the part of READ -COIiOR as illustrated in Figs. 10A and 
10B is done as shown in block C. of Fig. 10A. Returning 
to decision diamond 'G; if the status value of the 
current point is equal to FILL, then the module READ 
COLOR moves to the next point on ; the same row as the 
current point, as shown in block J., Decision diamond K 
then asks if the ' column coordinate, of . the next point is 
less than or equal to the column coordinate of the next 
perimeter point in the ^ perimeter buffer. If it is, 
then* the -next point is identified as an interior point 
as ' shown in block L. After identifying the interior 
points-'df an object- as shown, in block L of Fig. 10B, 
the gray level value of each interior point may be 
determined, and the object may be characterized by 
using- the gray -level values as described below. In 
addition, a predetermined gray level may be assigned to 
each interior poirtt to- label the -object. After the 
next point is ^identified r as the interior point as shown 
in bldck L, -the Idop through block : J and diamond K 
continues until ^the { column coordinate of the next point 
is greater than - the column coordinate .of the next 
perimeter 'point in the perimeter buffer. Then the 
current point is set to the next perimeter point in the 
buffer as shown in block M . The loop. through B - M 
until the current point is beyond the last point in the 
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buffer,, at. which point; the part of RELAD COLOR as 
illustrated in figs., .1 OA and 10B is done, as shown in 
block C of Fig. 10A. , v 

As noted; above, the method of the first embodiment 
of the present invention further includes the steps of 
determining the gray level value of each interior point 
and characterizing , the object by using the gray level 
values of the. interior points. The characterizing step 
may comprise determining the texture of the object. 
Alternatively, or in .addition, the characterizing step 
may comprise .determining the color of the object. 
Alternatively, or in addition, the characterizing step 
may comprise determining any patterns which are present 
in the object. . — L - , 

, . When color is us,ed to characterize the object, or 
as a predetermined attribute value to validate the 
object-, as will be described below, the method of the 
present invention includes the sub-step of calculating 
at .least one color parameter for at least one interior 
point ,cf the candidate object. By color parameter, it 
is meant any parameter which is related to the color 
content of the object. More specifically, three common 
parameters used in color theory are hue, saturation and 
lightness as defined, in "Color Science, Concepts and 
Methods,- Quantitative Data and Formulae", by G. 
Wyszecfci and W.. S. Stile, John Wiley & Sons (1967) . In 
a preferred application ..of the first embodiment, only 
the color parameter, -.hjie., is calculated. However, it 
is obvious that any color parameter or combination of 
color parameters could be used to identify or 
characterize a. candidate object by its color content. 

The gray level of an interior point may be 
described using "RGB" information of standard color 
theory, with R designating red content, G designating 
green content, and B designating blue content. Shown 
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in Fig. 11A is a description of RGB color space, and 
shown "in Fig i 'llB is a simplification of that space, 
known as the RGB hue circle. A simple way to calculate 
a relative hue parameter, known as hue angle, is 
illustrated with reference to Figs. 11A and 11B. The 
value of the hue angle of an object can range from 0° - 
360°, as it is'an angle 6 inthe RGB hue circle which 
is a measure of the relative 1 admixture of red, green 
and blue colors. The angle, $ is -a selected "offset 
angle" and is an Adjustable parameter which can be 
optimized for a specif ic application The angle $ is 
chosen such that for the problem of interest, none of 
the hue angles of the valid object comet too close to 0° 
to 360°. The reason for avoiding the 0° (or 360°) angle 
is due to th^ discontinuity in the value of -the hue 
angle for this arigl'fe\ 

the hue angle of each interior point in an object 
can be calculated. The hue angle 9 calculated from the 
RGB hue circle using" the following equations : 

a - Rcos ($) + Gcos (120 + <J» + Bcos{240 + ' <J» (1) 
b = Rsin(<|>) + Gsin(120 + <J>) (2) 
+ Bsin(240 + <|>)hue angle « tan" 1 (b/a) 

Where : 

<t> « adjustable offset angle 

r « gray level of a point in 'the image when 
the image is viewed through a red "filter. 

B =* gray level of a point in the image when 
the image is viewed through a : blue filter. 

g" = gray level 1 of a point in the image when 
the image is 'viewed through a green filter. 

- a/ b w projections of the R, G, B vector onto 
the hue circle. 

The mean hue is derived by averaging the hue 
angle for each interior point over the object. 
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- The hue magnitude, which is the length of the 
hue angle vector in the; RGB hue circle, is a measure of 
how much color an object has and is. calculated as: 



Hue: magnitude . = a 2 + b 2 



(3) 



The .mean . hue angle and the hue magnitude can be 
used to determine the hue content of an object. These 
parameters used to. characterize an object can be 

1Q compared with predetermined attribute values in order 

to identify an .object, as will be fully explained below 
with respect to the second embodiment. 

In a specific application of the first embodiment 
of the present invention-, the concept of entropic 

15 thresholding is employed. , , The concept of entropy in 
information theory for : signal processing was first 
proposed by Shannon in an article entitled "A 
Mathematical Theory of 'Communication", Bell System . 
Technology J. , Vol.;- 27, July, 1948, pp. 379-423. 

20 .Shannon : showed: ,that the entropy function: 

- n 

H(p lf p 2 , pn) = - X %p k In p k (4) 

• - * --Jc-l-.j * 

uniquely satisfies the following, three properties: 
25 (ai) H (Pi,P2/ • --Pn> is a maximum for: 

p k 1/n for k » 1, -. . .n; 

(b) -H(AB) =^H(A) +-H A {B), where A and B are two 
finite partitions and H A (B) is the conditional entropy 
of partition B given' partition A; 
30 H(pi,p 2 , . ...p h ,0) - H(p x ,p 2 r . • .p n > < 5 > 

in addition, H^fl/n, . . .1/n). = In n. (6) 
The idea of using entropy to analyze a gray level 
histogram of an, image was originally proposed by Pun in 
an article entitled "Entropic Thresholding, a New 



SUBSTITUTE SHEET 



WO 94/16402 21 PCTAJS93/02736 

Approach"; Comp-; Graphics and Image Proc, Vol. 16, 
1981; pp. 210-239^ • The- entropy analysis of Pun was 
further refined -by Kapur et al . in an article entitled 
"A New Method for Grey-Level Picture Thresholding Using 
5 the Entropy of the 'Histogram"-, Comp- Graphics and 

Image. Proc. 29, 1985, pp. 273-285. As shown by Pun 
and refined by Kapur, trie- coricept of entropy can be 
extended to two dimensions if the gray level histogram 
'of the image is iised r to define a probability 
10 distribution: "*"* 

Ps " f s / N for ■'S-l/.-.,Ng ray (7) 
where f s - frequency of gray level s 

N - # pixels in image 

N gray ~ :# gray levels 
15 It follows that the jent ropy function. of a. histogram 
describing ah image with a uniform gray level 
distribution is at a' maximum. < The. more peaks in the 
distribution/ the ' lower the . entropy.. 

In this specific application :of .the first 
20 embodiment, where -ehtropic thresholding is employed, 

the searching step includes the sub-steps of generating 
a gray level histogram of the image, where the gray 
level histogram has an entropy function and 
entropically selecting a threshold gray level such that 
25* the entropy function of the histogram is maximized. A 
module, HISTOGRAM,, is used to generate the gray level 
histogram of ..the ..region of interest of the image. The 
steps fbr^generating the.gray level histogram are shown 
1 in the. /flow ^hart^of Fig. >12. As shown in block A of 
30 Fig. 12, HISTOGRAM., first calculates a histogram of the 
region of interest o,f Jthe image. It then calculates 
the values to be- u^ed subsequently in the calculation 
of the- entropy function, H s , for each gray level, s, as 
shown in -block B of Fig. 12. The results of this 
35 calculation are stored in memory as shown in block C. 
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This ensures that, , for. subsequent calculations of the 
entropic threshold gray, level , only a simple look-up 
operation is required. 

The entropic thresholding application of the 
5 method according to first embodiment of the present 
invention also includes the step of entropically 
selecting a threshold gray level value such that the 
entropy function of the histogram is maximized. This 
step is performed by the ENTROPY ; module as shown in 
10 Fig. 13. As shown in block A of Fig. 13, the first 
step in maximizing the entropy function of the 
histogram is to initialize the maximum entropy function 
to a minimum value. 

The step of entropically selecting a threshold 
15 gray level includes the sub-step of sequentially 

partitioning the gray level histogram at each gray 
level value into a first partition and a second 
partition. To illustrate the simple case where a 
single, simple object in a varying background is 
20 identified, a gray level histogram of an image is shown 
in Fig. 14. The first and second partitions are shown ? 
in the histogram of Fig. 14, where the gray level 
values of the background are represented by a first 
partition A, and the gray levels of the valid object 
25 are represented by a second partition B. In the 

ENTROPY module, the partitioned threshold gray level 
value is initialized to a minimum value as shown in 
block B of Fig. 13"*. 

The step of entropically selecting a threshold 
30 gray level also includes- the sub-step of computing the 
entropy function for each partition, where the total 
• entropy function of the histogram is defined as the sum 
of the entropy , function H s (A) of first partition, A, 
and the entropy function H S (B) of second partition, B. 
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This step' is shown in block C of Fig. 13 and is 
mathematically "expressed as follows : - 

For a given threshold' gray level value, s, : 

5 .. . ' H S (A) = - I g ln(ff) ' (8) 

i=l 

with Pi rff, P s .= , ^ . X^i . . {9) 

. ' rh US/ = -f- 1 1 .with> 5 = ; ffi do) 



10 



i-1 



1 . O 



i-1 



So,. H S (A) .- - £ t£) In <^) 



(11) 



-1 5 



= 7T . Sf^i In, r± + in W. 



s i=l 



15 Similarly, * S (B) - jjjr S'/i ^ fi + In N s ' (12) 

i=s+l 



. . where N s ' = N - N s 

.... an.£f ... ..• --• • „ 

20 H s (A)+H s (B)-ln N s +ln Ng'-^-Xfi in fi-j^T ^ffiln ^ <13) 

" The sum H S (A) + H s (B) represents the total entropy 
function of the gray level histogram of the image. The 
maximum entropic threshold gray level value is the 
25 value of s which maximizes the total entropy function. 
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Decision diamond, D of Fig. .13 asks whether the 
entropy function of the histogram is greater than the 
maximum entropy function as initialized, in block A. If 
it , is, then the maximum entropy function is updated 
5 using the partitioned threshold gray level as shown in 
block E of Fig. 13. The maximum entropic threshold 
gray level value .is then set to the partitioned 
threshold , gray level value as shown in block F. After 
the maximum entropic threshold gray level value has 

10 .been set, or if the entropic function of the histogram 
is not greater than the maximum entropy function, then 
decision diamond G of the ENTROPY module as illustrated 
in..Fig. f 13 asks whether the partitioned threshold gray 
level value equals the maximum threshold gray level 
..15. value. If so, the maximum entropic threshold gray 

level value is returned as shown in block H of Fig. 13. 
If not, then the partitioned threshold gray level value 
; is incremented as illustrated in block I of Fig. 13, 
and the incremented partitioned threshold gray level 

20 value is returned to block C, where the entropy 

function of the incremented, partitioned threshold gray* 
level value is computed. The loop through C - G is 
repeated until the partitioned threshold gray level 
value equals the maximum threshold gray level value, at 

25. • which point the, maximum entropic threshold gray level 
value is. returned as .shown in block H.. 

The probability distributions are renormalized in 
H 3 (A) and H s (B) to include only the gray level values 
within each of the partitions. With this 

30 renormalization, the. maximum entropy function occurs 
. right at the edge of the object peak in the gray level 
histogram as shown at. T in Fig. 14. Thus, a new 
threshold gray level value is selected such that the 
entropy function of the histogram is maximized. With 

35 this maximum choice of threshold for the simple case as 
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illustrated in Fig. 14, the renormalized distribution 
of the background becomes the least peaky and the most 
uniform". '* The total entropy function of the histogram 
is dominated by the entropy function of the background, 
5 since the number of 'gray" level values in the background 
partition is much larger than the 'number of gray level 
values in the object partition. 

In the application of the first bhtbodiment where 
the entropic threshold gray level i$ Maximized, the 

10 searching step further 'includes " the 3ub-step of 

scanning the image foir at least one object using the 
entropically selected threshold gray l£vel value . More 
specifically, the portion of the image scanned may 
comprise a plurality of* pixels, and 'each pixel has a 

15 gray level value less than the upper delimiter plus an 
increment The upper delimiter also his the notation 
MAX. The increment is equal to 'the difference between 
MAX, the maximum gray level value of t&e region being 
searched, and MIN, the minimum gray level value of the 

2 0 region being searched, resulting in a hew "maximum gray 
level value, Gray level^^: 

Gray level^ = 2 X MAX - MIN' (14) 

25 Regions in the image where the gray level value exceeds 
gray level^ are ignored in the search. 

T^he searching step further includes the sub-step 
of tracing the candidate ! object havirig boundary gray 
level values determined' by* the 1 entr'opically selected 

30 threshold gray level Value. In this" application, the 

searching step is performed by a module' SEARCH IMAGE as 
shown in Fig. 15, a module FIND OBJECT' as shown in Fig. 
16 f and" the module GET PERIM as shown in Fig. 3. 

As illustrated by block A of Fig. 15, the first 

35 step' is to initialize the search' position . The module . 
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J - • ■ SEARCH IMAGE searches the,,region of interest with the 
; current entropically. selected, threshold, gray level 

value. Decision diamond B, of Fig. , 15 then asks whether 
'the search position is at the end of , the scan. If so, 
5 'SEARCH IMAGE is done. 

If the' search position is not at the end of the 
scan, then the module SEARCH IMAGE searches the region 
of interest with the current entropically selected 
threshold gray level .value until it finds a point which 
10 has a -gray Level value- exceeding the entropically 
' • selected- threshold gray ,level value using a module FIND 
OBJECT. : To identify objects darker than the 
background; the image is inverted immediately after it 
has been generated. Such a point might be. the first 
15 : ; point'' of a new object..: -Decision diamond D of Fig. 14 
asks whether a new object has been found using the 
' module FIND 'OBJECT. If - ; the object has not already been 
' : ' traced' in the current search, the module SEARCH IMAGE 
proceeds to trace the object by running the module GET 
20 PERIM, as shown by block E of Fig. 15. The module GET 
• PER-IM is described in, detail above with respect to Fig.* 
3 . After the object: has been . traced,, the search 
position is incremented as illustrated in block F of 
Fig. 15. The loop through B - F is continued until the 
25 ' module SEARCH IMAGE is .at the end of the. search as 

indicated by decision diamond. B. Alternatively, if a 
' new object has. -not -been, found as indicated by decision 
diamond D, then the search position is incremented as 
illustrated in block- F, ,. thus, bypassing the tracing 
30 step, and the loop through B .-. I is continued until 
SEARCH IMAGE is at the end of the search. 

The steps of the module FIND OBJECT are 
illustrated in Fig. 16. The first step in FIND OBJECT 
is to initialize the search position to the current 
35 location of the image being searched as shown in block 
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A. Decision diamond B then asks whether the search 
position is inside the object. .If so, then the search 
position is' incremented as • illustrated by block C, and 
decisidn diamond b asks whether the FIND OBJECT is at 
the end of its search. If so, then no new. object is 
found as/ indicated in block E. If not, then decision 
diamond 3 asks whether the ■ incremented search position 
is inside the bb ject . ■• This process, of looping through 
B - E continues until the search position . i.s not inside 
the object. At this point., decision diamond F asks 
whether a next object has been found. If not, then the 
search position is incremented as. illustrated in block 
G "of Fig. 15, and decision diamond H, asks whether 
SEARCH IMAGE module is: at the end of its search. If 
15 so. Vhen no new object found is returned as, indicated 
' by block I. If not, then ..decision diamond F again asks 
whether a next object has 'been found using the 
incremented search position.. This.pro.cess of looping 
through F - I continues until- a next, object has been 
20 round: Decision diamond J asks whether the object 

" which has teen found has already been .traced. If so, 
the new object found is returned , as indicated by block 
K. If the object which has been found has not already 
been traced, then the search position is updated as 
25 illustrated by block L, and a new -object, found is 
returned as indicated by., block M of Fig. 16. By 
running SEARCH IMAGE, FIND OBJECT, and GET PERIM as 
described above, the =object returned in block M of Fig. 
16 'is then traced to determine the interior points 
30 thereof as shown in ■ block C of Fig. 1. 

In accordance with a second .embodiment of the 
present invention, there is provided a method of 
• identifying at least one valid object having at least 
one predetermined attribute value in a background. The 
35 method comprises generating an image of the object and 
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- the background. Fig. .17 is, a - b,loclc diagram showing the 
overall method of the second embodiment of the present 
invention . ;An image is generated ..as shown, in block A 
of Fig. 17. .As in the -.previous embodiment/ the 
5 hardware used to implement the - method of the second 
embodiment* :of the present invention must first be 
initialized when , the image is generated. . , The image of 
the* object and 1 the background ; may be generated by a 
' camera.- As with ■. the previous embodiment, t a CCD camera 

10 is used, although any... type of camera, may be used 

without' departing from, the general .principles of the 
present "invention . The 1 image is then digitized and 
stored by a frame grabber or a video digitizer. 

The method of identifying at least one valid 

15 object -of the present invention : also includes the step 
of searching the image for at. least one candidate 
object using at least. one automatically calculated 
' threshold, wherein the candidate .ob ject has at least 
one candidate object .attribute value. This step is 

20 shown in block B of Fig. 17. In the second embodiment 
of the present invention, the entire object may be 
searched. However, in instances where, it is known that 
an identifying feature of an object. in question lies in 
a particular portion of an object, the searching step 

25 may comprise searching only a portion of. the object. 
The automatically calculated:- threshold may be 
determined in a variety of -ways > For instance, the 
automatically calculated threshold is determined using 
a histogram as described above * in the first embodiment, 

30 or a co-occurrence matrix, as described in Pal and Pal 
in an article entitled "Entropic Thresholding", Signal 
Processing, Vol, 16, -1989, pp^.9.7 - 108. 

The ability to automatically threshold an image in 
order to identify and validate candidate objects makes 

35 it possible to perform "on-line" image analysis for 
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monitoring and controlling manufacturing and industrial 
- processes. This capability would not be .possible in 
general 'if a manual thresholding scheme is used, since 
the threshold (or thresholds) would have to be adjusted 
5 on an image-by-image -basis . Performing unsupervised 
'"'automatic object identification enables ..the ; use of 
; image analysis techniques' "for a wide range of 

industrial applications.^ In environments where the 
' ■ background varies unpredictably, where many methods 
lo' which do not use ah! ;autpraatically calculated threshold 
fail, the recursive' calculation of an automatic 
threshold, as describe'd' below, is particularly 
powerful. . ■ • 

The method of identifying. at least one valid 
15 object of ■■■the - present invention further comprises the 
step of determining the perimeter points of the 
' candidate -objects This -step is ydpne by. gracing the 
object as shown in block C of .Fig . ,. : 17 - _ The coordinates 
of each perimeter point of the pjbject are determined by 
20 the module GET PERIM as. described above with respect to 
Fig. 3. :;r; As in the first embodiment, the step of 
determining the ^perimeter points may comprise tracing 
the -object sequentially in the clockwise direction, or 
tracirig 'the object sequentially in the counterclockwise 

25- direction. • - ' - 

• ' ^ fhe method Of identifying; at r ieast one valid 
object of the present -invention further comprises the 
step ; of creating-a perimeter buffer. This step is 
shown gerierally<;by block -D of Fig. 17 and is performed 

30 in -the-module, LOAD BUFFER, ; as shown and described 
above with respect 'to Figs . 6A and 6B . 

The method of identifying at least one valid 
object of the present invention also comprises the step 
of sorting the perimeter points, in the perimeter buffer 

35 in a predetermined order to create a sorted perimeter 
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. :■ buffer. . This, step is shown in- -block. E of Fig. 17. The 
sorting ..step is performed as described .above with 
respect, to the first- embodiment for. tracing the object 
in either the clockwise- or the counterclockwise 
5 ( direction. t • 

The method^of identifying. valid objects of the 
present invention, also comprises the. step of assigning 
to each- perimeter, point a status variable having one of 
a first and, a second value. This step is shown 
10 generally in block. F ,of ... Fig.. 17 and is performed in the 
module READ COLOR as shown and described above with 
respect, to Figs. 7 and 10. 

, The ; method. of identifying at least one valid 
- obiect in accordance with :! the second embodiment of the 
15 present invention further comprises the step of 

designating as the interior points of the object each 
. point . cr. the same line segment as the perimeter point 

being examined and lying between the examined perimeter 
• point and the next perimeter point. in the sorted 
20 perimeter buffer, where the examined perimeter point is 
assigned the first value, FILL, of the status variable. 
This step is shown generally in block G.of Fig. 17 and 
is performed in the module READ. COLOR as described 
i. above with respect to Figs., 7 and, 10 for any of the 
,25 implementations of , the first embodiment. 

The method of identifying^ at least one valid 
object in accordance with -the second embodiment of the 
present invention further- comprises, the. step of 
labeling the object by assigning a predetermined gray- 
30. level value to* each designated interior point. This 

labeling, is done in the. module READ- COLOR, as shown and 
described- above with : . respect to block L in Fig. 10B. 

The method of identifying at least one valid 
object of the present invention further comprises the 
2 5 step of checking for redundant perimeter points. This 
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step is illustrated "in the flow chart of Fig. 10A by 
' decision d£amond :: E and block F, as described above. 
When this redundant' interior point checking is 
performed, the step of designating all points lying 
5 between each redundant perimeter point and the next 
perimeter point ' on' the same line segment as the 
interior points of the 5 object occurs only when all 
instances of each redundant" perimeter point are 
assigned the first 'value* of the status variable (FILL) , 
10* as described above with respect to the first 
embodiment. ' ' ' ' ' " ~ r 

The method of the second embodiment of the present 
invention further includes* the steps of determining the 
gray level value of each interior point and 
15 characterizing the 'object -by using the gray level 

values of the interior points. The characterizing -step 
may comprise determining the texture of the object. 
Alternatively, or in addition, the characterizing step 
may comprise determining the color of the object. 
20 Alternatively, or in addition, 'the characterizing step 
may comprise determining any patterns which are present 
in the object. J ' ' 

The method for identifying at least one valid * 
object of the present invention also comprises the step 
25 of characterizing/the candidate object by. the candidate 
" object attribute value. This' step could be the same as 
the' step of characterizing the object by using the gray 
level values' of J the interior' points, such as when the 
object attribute values use the* interior point 
39 information. -In one-instance, the candidate object 
attribute value maybe color. Alternatively, or in 
addition, the candidate object attribute value may be 
* the texture of the object, or in another instance, a 
pattern in the object. 
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The method of identifying at least one valid 
object of the present invention also comprises the step 
of validating the candidate object having the valid 
object predetermined attribute value to identify the 
5 valid object. This step is shown in block I of Fig. 
17. The validating step includes the sub-steps of 
calculating the candidate object attribute values and 
. . , . comparing the candidate object attribute values to the 
valid object predetermined attribute values to validate 
- . 10 candidate, objects. 'The calculating sub-step may 

further include the sub-step of storing the candidate 
. -object attribute values. The validating step is 

performed by a module,* COL FILT, as 'shown and described 
. , with respect to Fig. 18'.*" In this embodiment of the 
15 , present .invention, COL FILT uses only size and color as 
valid object .predetermined attribute values. In 
general, other attribute values may be used for the 
valid object predetermined attribute values. 

The method pf the second embodiment of the present 
... 20 , invention further .includes the step of filtering the 
image. The filtering step is performed by the module 
COLFILT for either inhomogeneous or inhomogeneous 
objects. The filter in step tor filtering inhomogenous 
objects comprises three conditions under which objects 
• - 25 . should.be, deleted. . The first condition deletes objects 
. when an. object has an area less than a predetermined 
minimum, area. . The second condition delates objects 
when objects when the pbject has a mean hue angle less 
than a predetermined. minimum and a hue magnitude 
30, greater than a predetermined minimum 1 . The third 

condition deletes objects when an object has a mean hue 
ancle greater than a predetermined 'maximum and has a 
hue .magnitude, greater than a predetermined minimum. 
Hue angle and hue magnitude are calculated as described 
3 5 above in equations (1) and (2) above. 
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As shown in block A o£ Fig. 18', the first step of 
COL FILT fpr filtering objects is to advance to the 
initial object.. Decision diamond B' then asks whether 
COL FILT is beyond the last object. If it is, then the 
5 module ,is done as shown in block CV If it is not, then 
decision diamond D asks whether the object has an area 
greater than a predetermined minimum area. If the 
answer is yes, then the object is deleted as shown in 
block E, and the module advances to' the next object as 

10 shown in block F. Decision diamond B then asks whether 
COL FILT is beyond the last object, andthe loop 
through B - F continues until the object' area is not 
less than the predetermined minimum area. At this 
.point, decision diamond G asks whether the mean hue 

15 angle of the object Is less than" a predetermined 

minimum value, MINHUE, and whether the hue magnitude of 
the object is greater than a predetermined minimum 
value, MINMAG. If the answer to both these questions 
is yes, the object is deleted 'as shown in block E, and 

20 the module is advanced to the next object as shown in 

block F. The loope through 'A and B continues until COL 
FILT is beyond the last object, at which point it is 
done as shown in block c: If COL FILT is not beyond 
the, last object, the loop through A - G continues until 

,2-5. the object hue is greater than or equal to MINHUE, or 
... the object hue magnitude is' less than or equal to 

MINMAG. Then decision diamond^H asks whether the mean 
hue angle of the object is greater than a predetermined 
maximum value, MAXHUE and whether the ■ hue magnitude of 

30 the object is greater than the predetermined minimum 
value, MINMAG. If the answer to both these questions 
is yes, the object is deleted as shown in block E , and 
the module is advanced to the next 'object as shown in 
block F. The loop through A and B continues until COL 

25 FILT is beyond the last object, at which point: it is 



BNSDOCID: <WO 9416402A1 I > 



SUBSTITUTE SHEET 



WO 94/16402 



40 



PCT/US93/02736 



.done as sho.wn in block C. If ..COL FIL.T is not beyond 
the A Iast object, .the loop through A - H continues until 
the object hue magnitude . is less than or equal to the 
predetermined maximum .value, MAXHUE or the object hue 
5 magnitude is less than o.r equal ta the predetermined 
minimum value; MINMAG . . . If the answer to either of 
these . questions is no, the object is r retained, and the 
. , . module, advances £9. the next object as shown in block: F. 
The .loop through B - H continues until the module is 

10, beyond the last object,.. . The module is then done as 
shown in block C. ; - 

■ The method of . the second embodiment of the present 
invention- employs., driyer. and a kernel for validating 
, candidate objects. The. driver stores the attribute 

15 . values, of the valid object, where each value represents 
the definition of a valid, object, e.g., color, edge 
-contrasts area, snape, etc. The driver of the present 
invention ;is specific to, a given application. In an 
object-oriented,, environment, it is straight-forward in 

20 many instances to describe an object via a list of 

attributes such as, size, „shape, color, etc. For more 
complex obje_cts, where, a. simple .parametric description 
might not* be possible, one could use a neural network 
in the driver to identify the object. Parameters 

25 ' rderived- firom the candidate ..object can be fed into the 
neural network, which has been trained to recognize 
specif i-c. -objects At this point, the architecture of 
. the present invention ,begins t to resemble a neural 
vision architecture where there is a feedback loop 

30.. between the brain, and the eye. In the present 

invention, a high-order driver is. intertwined with a 
lower-order kernel. In this case, a more complex 
: description of the object is used to drive the 
searching process, which in turn identifies further 

35 candidate objects. 
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The driver drives the kernel: The kernel performs 
several functions .'' It automatically calculates an 
entropically selected threshold gray level value, 
searches the image and calculates the attribute values 
5 for a candidate object. In addition, it performs a 
validity check on candidate objects by comparing the 
attribute values of the' candidate objects with the 
predetermined attribute values for the valid objects, 
which, as noted above', "are contained- in the driver. It 
10 also performs a redundancy check to prevent multiple 
identification of a valid object. 

" : According to a first,- specific application of the 
second embodiment of the^ present invention, entropic 
thresholding is employed for searching the image. In 

15 this first', specif ic" application, the ^ step of searching 
the image includes the sub^-step of- generating a gray 
level histogram of the image . The gray level histogram 
has" ah entropy function/' The 1 module", HISTOGRAM as 
described above "with 5 re'spect '-to Fig. -12,' is used to 

20 ' generate' the gray level histogram of the' region of 

interest of the image . .The step of searching the image 
also' Includes the" sub-ste'p of Entropically selecting a 
thresnold gray level value such that the entropy 
' function of the histogram is maximized. This step is 
■ _ 25 performed by the ENTROPY module as described above with 
respect to Fig. 13. 

' :: ' " According' td the ¥htropic thresholding application 
of the second enibo^imerity the searching step further 
' includes the sub-st&p :; -bf° Scanning a portion of the 
30 image" "tor at lea'st" one' candidate object using the 

entropically selected "threshold gray level value. The 
" searching step further includes the sub-step of tracing 
the candidate' object having boundary gray levels 
determined by the entropically selected threshold gzay 
25 level. In this implementation, the searching step is 
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performed by the module SEARCH . IMAGE , as described above 
with respect to Fig. 15, the module FIND OBJECT as 
described above with respect to Fig. 16, and a module 
TRACE OBJECT as described" in Figs. 19A and 19B. The 
5 basic principles of the TRACE OBJECT module are similar 
to those described in "Digital Image Processing" by 
Rafael C. Gonzalez and Paul Wihtz, • Second Ed., Addison- 
Wesley Publishing Company , Reading', Massachusetts 
(1987) . * • - * 

10 ' As shown in block A of Fig. 19A, the first step in 

the TRACE OBJECT module is to initialize the candidate 
object attribute values. The TRACE OBJECT module then 
asks in decision diamond B whether a neighboring 
perimeter point has been found. If not,' the traced 

15 object is invalid as illustrated by block C. If the 
neighboring perimeter point has been found, then 
decision diamond D asks whether the TRACE OBJECT module 
is at the first perimeter point "of the candidate 
object. If not, then the candidate object attribute 

20 values ' are updated as illustrated in block E of Fig. 
19A. The loop through B - E is then repeated, using 
the updated candidate object attribute "values, until 
the TRACE OBJECT module is at the first perimeter point 
of the candidate object. . The center of mass coordinate 

25 is then calculated as shown in block F of Fig. 19A. 
Decision diamond G then asks if the candidate object 
area is too large. If it is, the traced object is 
invalid as indicated by block H of Fig. 19A. 

If the candidate object area is not too large, 

30 then a shape factor is calculated as shown in block I 
in Fig. 19B. The definition of the shape factor may 
vary, depending oh the geometry of the object being 
identified. For instance, the definition of the shape 
factor for circular objects is: 
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Shape - Factor = >'l : ~* ' < 15 > 

where: P. Is the perimeter of a candidate object; and 
A is the area of the candidate object. 

5 

TRACE .OBJECT then checks if the shape factor is within 
a ..predetermined range as contained in the driver as 
shown in decision diamond J in Fig. 19B. If the shape 
factor does not fall within the predetermined range, 

10 thenthe traced object is invalid as illustrated by 

block K of Fig. 19B. If the shape factor falls within 
the predetermined range, then the candidate object is 
.added to the list of valid objects maintained by the 
kernel as shown in block. L. ....... 

15 After all the candidate object have been traced in 

the current search, the module CHK GRAY as shown in 
Fig. .20 is called to check whether the candidate 
objects. are relatively lighter than the background. 
CHK GRAY is run after decision diamond C of Fig. 15, 

20 when the search position is at the end of the scan. As 
shown in block A of Fig. 20, the first step in the CHK 
GRAY module is to advance to the first candidate object 
found in the current search. Decision diamond B of 
Fig. 20 asks whether the candidate object is the last 
" 25. , object in the list of candidate objects. If it is, the 
module stops running as shown by oval C. If the 
candidate object is not the last object in the list of 
candidate objects,., then the average exterior gray level 
is calculated as illustrated in block D. Decision 

30 diamond E then asks whether the gray level of the 

center of mass is greater than the average exterior 
gray level of the four exterior points (i.e., the top, 
bottom, left and right points) surrounding the extremum 
points of the object. The exterior points are those 
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points in the background. whi,ch,are immediate neighbors 
to the extrernum points .of .the, object. ■ If. not, the 
object is deleted as shown in block-F of Fig. 20. If 
the gray level center of .mass is greater than the 
5 average exterior gray level,, then the candidate object 
is retained, and ( the QHK GRAY module advances to the 
next candidate object as. shown in block, G. The CHK 
GRAY module, then returns to decision, diamond B to ask 
whether the candidate object is the last object. The 
.0 lopp, as. shown, in ^ - 6 is repeated. for . the next 

candidate object, until the next candidate object is the 
last candidate object, at which point . CHK GRAY stops 
. running. ... . As f noted above,, the module, CHK GRAY may be 
rurx i^tect. objects darker than the background. In 
5 this .case f the image is ..initially inverted prior to 
performing th,e step of generating the image of the 
, object and the background... 

The method as described thus far can be referred 
to as , ; a screening process. For example, : ,it can be used 
0 , to screen for the presence of pathological bacteria in # 
. fopd or in blood or soil samples. A screening process 
results, in a yes - no. answer; absolute quantitation is 
not necessary. For a more stringent identification 
process, it is necessary to apply the method of the 
5 entrppic thresholding application of the- , second ; 
embodiment , of the present . invention recursively as 
described below. . v sv ^ . .. . 

The recursive .version of the first,, specific 
. application of the second embodiment may further 
) comprise the steps qfi -subdividing the gray level. 

histogram into an .upper histogram and a lower histogram 
using the entropic threshold gray level which was 
selected to maximize the entropy function of the 
histogram as an upper delimiter and a, lower delimiter. 
The selection, searching, validating and- subdividing 
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steps are* recursively repeated for each of the upper 
and lower histograms. The repetition of the selection 
step select's a ' next successive entropic threshold gray 
level'; thereby recursively partitioning the gray level 
5 histogram to identify the valid objects " until a 

predetermined minimum nuhiber'of new valid objects is 
identified'. In the preferred implementation of this 
application, the pre'dete'rhiined minimum number is zero. 
However/ there may be 'ckses where the predetermined 
10 number is greater than zero, sxich : as when a complete 
identification is' hot required'. 1 

Figs. 21A - 21C illustrate the concept of 
subdividing ' a histogram * into an upper' histogram and a 
lower histogram. ' An' oiriglnal Histogram is shown in 
15 Fi'gv THRESH; as * shown at T in ; Fig'. 21A, is the 

• entrdpiially selected threshold gra:y' level for the "gray 
level histogram corresponding 'to the gray level region 
between 1 -the rnrinimum - gray level be : ing searched and the 
maximun- gray level beiiig searched. For the original 
20 histogram as shbwn in Tig. 21A, 'the' minimum gray level 
being- Searched is zero and the maximum gray level being 
searched is MAX. THRESH HI, as shown at B, is the 
entropically selected : threshold gray level for the ~gray 
levePhistogram corresponding to the" gray level region 
. 25* ■_ -"between THPX5H and ; MAX . THRESH LO, as shown at A, is 
the -enttoplcaliy- selected ^threshold gray level for the 
gray level histogram corresponding to the gray level 
region between ier6°ahd : THRESH. 

; " -Accordirig :J tO' th^'iecursiive version of the first 
30 specific ' application, "the' subdividing, selection, 

searching and 'validating -steps are then recursively 
repeated. 3y recursion is meant the process of 
continuously dividing a histogram into upper and lower 
histograms, searching each upp^r histogram, which upper 
35 histogram is itself continuously divided into upper and 
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lower histograms,, for, new valid objects until the 
number of new valid objects found in an upper histogram 
is less than or equal to a predetermined minimum 
number, and subsequently searching each lower histogram 

. 5 corresponding, to the most recently searched upper 

histogram, . which lower histogram is itself continuously 
divided^ into upper, and lower histograms ,. until the 
numbef of new, valid objects found, in a lower histogram 
is. less r than or equal to the predetermined minimum 

1.0 number . 

The upper histogram is shown. in Fig. 21B. The 
.repetition qf the subdividing step subdivides the upper 
, histogram, into. a next successive upper and lower 

histogram , as. shown in Fig. 21B. The repetition of the 
15 selection. step for the upper histogram selects a next 
upper successive entropic threshold gray level value, 
as shown at B in Fig. 21B. Thus, point B, which was 
THRESH HI in the original histogram, becomes the 
threshold for the upper histogram,- or NEXT UPPER 
20 THRESH. . In Fig. 21B, the minimum gray level value . 

. being searched is now THRESH and the maximum gray level 
.value being searcned is now MAX . The NEXT UPPER THRESH 
. HI, shown at C, is the entropically selected threshold 
gray level value for the gray level histogram 
2 5 corresponding . tp . the ..gray, level region . between B and 
.MAX. The NEXT UPPER .THRESH XQ,, shown, at : D, is the 
entropically selected threshold .gray level value for 
the gray level histogram corresponding to the gray 
level region between THRESH and B v The selection, 
30 searching, and validating steps are then repeated 

recursively using the next upper successive entropic 
threshold gray level value, B, as. the entropic 
threshold gray level value.. 

Fig. 21C shows the. lower histogram. The 
35 repetition of the subdividing step subdivides . the lower 
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histogram into a next successive upper and lower 
histogram as shown in Fig. 21C. The ^repetition of the 
selection step for the lower histogram selects a next 
lower successive' entropic threshold gray level value, 
5 as* shown at \A in fig- ^21C. Thus, ' point A, which was 
THRESH LO in this original histogram/ becomes the 
threshold for the partitioned lower histogram, or NEXT 
LOWER THRESH. In Fig. 21C, tie minimum gray level 
value being searched is " how zero" and' the maximum gray 

10 level value being searched is now THRESH. : The NEXT 
LOWER THRESH HI, stibwn at E, is 'the entropically 
seiedted threshold gray' level value for the gray level 
histogram corresponding to the gray ~ level region 
between A and THRESH. The NEXT 'LbWER THRESH LO, shown 

15 "at"F, is the entropically ' selected threshold gray level 
value for the ' gray 4 level histogram 1 corresponding to the 
gray level region between zero and A 'J ' The selection, 
searching and validating steps are then repeated 
recursively for the lower histogram using the next 

'20 lower successive entropic threshold gray level value, 
A, as the entropic threshold gray level value . 

The ANALYZE module as shown in Fig; 22 constitutes 
the core recursive kernel of the recursive version of 
: the second embodiment . * The ANALYZE module effectively 

25 zooms in on a specific region in gray level space to 
seai-ch for instancies of 1 'the candidate object and 
recursively partitions the histogram . The first step 
in the ANALYZE module as "shown in Fig. 22 is to 
calculate the 1 entropically selected threshold gray 

30 * level values ^HRESH, THRESH HI AND THRESH LO as 

described above and as shown in block A of Fig. 22. As 
shown in block B, the module SEARCH IMAGE is run using 
the gray level values contained in the upper histogram. 
Decision diamond C then asks whether the number of new 

25 valid objects found is greater than the predetermined 
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minimum , number , . . .If it is, then the module ANALYZE is 
run on the . upper , histogram recursively. If the number 
. of valid . objects^ found is not greater than the 

predetermined minimum number, then the module SEARCH 
5 t IMAGE is run again . using the gray level values 

.contained in the lower histogram as shown in block E. 
■ Decision, diamond F then asks whether the number of new 
valid .objects found is greater than the ? predetermined 
minimum number If.it is, then ANALYZE is run on the 

10- lower histogram recursively as shown in block G. If it 
is not, then ANALYZE stops running, and the valid 
. objects are returned as shown in block H of Fig. 22. 
With the present invention, there is some latitude in 
- . selecting the range of values of the number of 

15.,- .attributes to be checked for in the validation step 
during, the recursive process. 

According to the method of the first specific 
application of the second embodiment of the present 
invention, the validating step further includes the 

20 sub^step of checking for redundancies to prevent , 
multiple identification . of the valid object. Such 
redundancy checking is necessary since an object which 
has been recognized as valid after the current search 
, may have been recognized as a valid object in an 

25 earlier search. In order to perform the redundancy 
checking sub-step, valid objects are classified as 
either homogeneous or inhomogeneous. Inhomogenous 
objects are described in the example given below, where 
color colonies are. identified. Valid objects are also 

30 - further classified as either a relatively large valid 
.object or a small valid object. In addition, valid 
objects are classified as either contained in another 
valid object (interior valid objects), or never 
contained in another valid object (exterior valid 

25 objects) . 
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According to'' the method' of the first specific 
application of the second embodiment of the present 
invention, the * redundancy checking sub-step may be 
performed to 1 delete inhomogenebus valid objects. When 
5 it is, the method of the present invention further 

includes the sub-step of deleting the large object when 
the large object contains more than * one' small object. 
Also'^ when the redundancy checking- sub-step is 
performed to delete inhomogeneous valid objects , the 

10 method of the present invention also "includes the sub- 
steps of calculating the average edge contrast of the 
large and the small valid objects and deleting the 
object having the smaller edge contrast when the large 
object* contains only one 1 small object. 'These sub-steps 

15 are performed by a module'/ CHK LIST/ as shown in Figs. 
23A - D for inhomogeneous valid objects. 

As shown in block A : of Fig. 23A, the first step of 
the CHK LIST module for deleting inhomogeneous objects 
is to define the previous count as the number of valid 

20 objects found prior to the 'Current search. Then the 

tail object "is defined as the initial candidate object 
found in the current search as shown in block B. The 
object count is initialized to one as shown in block C, 
and" the head object is defined as the initial object in 

25 ' the total object"' list (i.e., the list of all objects 

found to date') as shown in block D. - Decision diamond E 
asks whether" the" bbj^dt ; count • is greater than the 
"previous "count. 

If the* objebt count is greater than the previous 

20 count, CHK LIST advances'to the first object in the 
total object list as shown' In block A of Fig. 23B. 
Decision diamond B of Fig. 23B asks if CHK LIST is at 
the last" object . If not, then decision diamond C asks 
whether the valid object is contained within another 

35 valid object. If so, the object status is set to the 
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t , status of the. object within, which it is contained as 
shown in block : D, and CHK LIST advances to the next 
object as shown in block E. Also, if the object is not 
contained within another object, then CHK LIST advances 
5 to the next object as shown in block E. The loop 

through B - E continues until the next object of block 
E is,. the last object, at which point CHK .LIST advances 
.to the first., object in the total object list as shown 
•in block F . The. object status attribute values for all 
10 the qbjects is set to r Vtrue " as shown in block G. 

"True" in this, context means valid, and "false" means 
invalid. Decision ;J diamond H then asks if CHK LIST is 
at the last , object... 

I-f /it is, CHK LIST i advances to the first object as 
.15 , shown in block A of Fig. 23C. Decision diamond B then 
asks again whether CHK LIST. is at the last object. If 
it is> ; then th^ total number of objects is counted as 
shown in block C, and the difference^ between the total 
number of .-objects ahd , the previous count is returned as 
■ 20 shown, in block D. If CHK LIST is not . at the. last , 
object, t decision diamond. E asks whether the object 
; status attribute value .is fals.e. ...If soothe object is 
deleted as shown in blpck. r F. I.f . not , then CHK LIST 
advances the object as shown in block G, and CHK LIST 
.25 . asks . again whether it is at the last object as shown in 
decision diamond B. The loop, through B,,, E, F f and G 
continues; until the advanced object of block G is the 
last object. : At. this ppint,. , the total number of 
objects is counted as shown ; in blpck C, and the 
30 . difference between the total number of objects and the 
previous count is returned, as shown in block D. 

Returning to decision diamond H in Fig. 23E, if 
CHK LIST is not.at the . last, object., at this point, then 
it goes to decision diamond I, which asks whether the 
35 object contains more than one valid object. If so, 
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then the object 1 status attribute value is set to false 
" as shown in block J, and CHK LIST advances to the next 
' '' object as shown in block K. CHK LIST then returns to 
decision* diamond H, asks whether it is at the last 
5 object' and continues this process' until the object does 
" not contain more than 'one- valid object . " Then decision 
•• diamond A of Fig. 23D asks if the object is the only 
' "object contained within another object . If not, then 
CHK LIST advances to the 'next object as shown in block 
10 K of Fig: 23B, arid the ' loop through H - K of Fig. 23B 
and A of Fig. 2 3D" is"' repeated until the object is the 
only object contained "within another object. If the 
object is the only object contained within another 
object, then decision " diamond- B asks whether the status 
15 attribute value' of the object 'which contains the object 
is false. If so, then CHK LIST advances to the next 
' " " object a ( s shown In block K of- Fig'! 23B> and the loop 

through H — K' of Fig. 23B and ! A - B of Fig. 23D is 
repeated until the' status attribute value- of the object 
20" which Contains' the object is -hot false. At this point, 
decision- diamond N asks whether-' the edge contrast of 
the object which contains another object is greater 
than the edge contrast of the object. If so, then CHK 
LIST sets the object status attribute value to false as 
• - 25 shown in block Dy it advances to the next object as 

shoW" in' block K iri r Fig. 23B, arid the- loop through H - 
K J of : Fig. 23% and-'A^C'bf Fig. 23D -is repeated until 
the edge contrast df the object which contains another 
object Is : not' greater^ than the edge contrast of the 
' 30 object contained in another object . - Then CHK LIST sets 
the status of the object which contains- the object to 
'false as shown in block E ofFig. 23D, and it advances 
to the : next object as shown in block K of Fig. 23D 
until it is at the last object. 
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Returning to decision diamond E in Fig. 23A, if 
the object count is not greater than the previous 
count, then decision diamond F asks if the head object 
is contained within another object. if so, then the 
5 head object is advanced as shown in block G, and the 
object count is incremented as shown in block H. 
De.cision diamond E again asks if the incremented object 
count is greater than the previous count. If so, CHK 
LIST advances to block A of Fig, 23B as explained 
10 above.. If the incremented count is not greater than 
the previous count, the loop through F, G, H and E in 
Fig. 23A is repeated until the head object is not 
contained within another object. Then CHK LIST 
s . advances to decision diamond I of Fig. 23A, which asks 
,15 if the tail object is the last object, or if the head 

object is contained within another object. If the tail 
object is the last object, or if the head object is 
contained within another object, then CHK LIST advances 
the head object as shown in block G, and the count is 
20 . incremented as shown in block H.* The loop through E, 
s[, I, G .and H is repeated until the tail object is not 
the last object or the head object is not contained 
within another object. Decision diamond" J then asks 
whether the tail object is contained within another 
25 object. If it is, then the tail object is advanced as 
shown in block K of Fig. 2 3A, and the loop through I, J 
and K is repeated until the tail object is not 
contained within another object. Then CHK LIST goes to 
the module SET STAT as shown in' Fig. 24 to set the 
30 status of the head and tail objects as shown in block L 
of Fig. 23A. " . \ 

The redundancy checking sub-step further includes 
the sub-steps of comparing the areas of a plurality cf 
valid objects and designating one of the valid objects 
3 5 as a large valid object and the other of the first and 
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second valid objects as a small valid object and 
determining whether the' small valid object is contained 
in the large valid object as defined by the four 
extremum points of the larger object for inhomogeneous 
objects. The module SET STAT as shown in Fig. 24 
performs these sub-steps for inhomogeneous objects. 
The first step of SET ' STAT as shown in decision diamond 
A of Fig. 24 is to ask whether 'the' head object is 
larger than the taii object. if' so, then the head 
object is defined as the large" valid object, and the 
tail object is defined as the small * valid object as 
shown in block B. If the head object is "not larger 
than the tail object, then the head object is defined 
as the small valid object, and the tail object is 
defined as the large valid object as shown in block C. 
Then decision diamond D asks whether the small object 
is contained within the large object. If not, then SET 
STAT is finished, as indicated by END oval E. If the 
small object is contained within the large object, then 
the large object type attribute value is set to a value 
indicating that it contains a small object as shown in 
block F . The type attribute value 'tells SET STAT 
whether an object is contained within another object or 
whether the object contains another object. Also, the 
small object type' attribute value is set to a value _ 
indicating that it is contained within a large object 
as. shown in block G. Finally, the large object status 
attribute value is incremented as shown in block H. 
SET STAT is then finished, as' indicated by the END oval 
I and returns' to block L of Fig. 23A. 

According to the first, specific application of 
the second embodiment., the redundancy checking sub-step 
may be performed to resolve redundancies in the 
homogeneous objects! When it is, the method of the 
present invention further includes the sub-steps of 
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calculating the edge cpntrast of the large and small 
valid objects and deleting the large object where the 
average edge contrast of the large object is less than 
the average edge contrast of the small object and is 
,5 less than a predetermined minimum edge contrast. The 

redundancy checking sub-step for resolving redundancies 
also ^includes the sub-steps of calculating the edge 
contrast of- the. large and small valid objects and 
deleting .the small object where the average edge 
. -10. contrast, of the large object is greater than the 
average edge, contrast, of the, small object^ and is 
greater than the predetermined minimum contrast. These 
sub-steps are performed using the module CHK LIST for 
homogeneous objects as illustrated by the flow charts 
15 of - Figs .- 25A and 25B,. T 

As. shown in block A of Fig. 25A, the first step of 
the CHK LIST module, when run to delete homogenous 
objects, is to define the previous count as the number 
. of valid, objects found prior to the current search. 
20 Then. the. tail object is defined. as the initial 

candidate, object found in the current search as shown 
in block B. The object count is initialized to one as 
.shown in block C, and the head object is ^defined as the 
. initial., ob ject in the total object list as shown in 
. 25, block D ( . The. object status attribute, value is. then set 
to true for all . objects . as shown in block, E. Decision 
diamond F asks whether the object count is greater than 
. the previous count. 

If the object .gount, is greater than the previous 
; . 30 count, CHK LIST advances to the initial object in the 
total object list . as .shown in block A of Fig. 25B. 
Decision diamond B of.Fig..25B asks if CHK LIST is at 
the last object. If. so, the total . number of objects is 
counted as shown in block C, and the difference between 
, 35 the total number of objects and the previous count is 
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' returned as shown in block D / If CHK 'LIST is not at 
the last object, then decision " diamond E asks whether 
the object status attribute value is 'false- If so, the 
object is deleted as shown in block F. If the object 
5 status is hot false, then object is advanced as shown 
in block G, and the CHK LIST module "asks again whether 
it is at the last object as shown in decision diamond 
B. This process continues until CHK LIST reaches the 
last object, at which point the' total number of objects 

10 is counted as shown in block C, and the difference 

between the" total number "of objects 'arid the previous 
count is returned aV bh&wn in block D." 

Returning to^ "deci'sioft diamond F in Fig. 25A, if 
the object count is' not greater 'tharv the previous 

15 count, then decision diamond' G of Fig. 25A ; asks if the 
status attribute value" of" the head object is false. If 
it is / then the' head object is advanced as shown in 
block and the count is ' incremented as shown in block 
I. Decision diamond F then asks if the incremented 

20 object count is greater than the previous count. If 
so, CHK LIST advances to block A of Fig. 23B as 
explained above. The loop through G, H and I in Fig. 
25A is repeated until the status of the object is not 
false. Then CHK LIST advances to decision diamond J of 

2 5 rig. 25A, which asks if the tail object is not the last 
object and if the" hail bbject status attribute value is 
true! The answer" to both these questions must be yes. 
If not, then CHK LIST advances the head object as shown 
in block H, and the v cdurit iS incremented as shown in 

30" block li' The loop through F, G, H, I and 'J is repeated 
until the tail object is the last 'object and the head 
object status 'attribute' value is true. Decision 
diamond K then asks whether the tail object status 
attribute value is true. If it is, then the edge 

35 status of the head and tail object is set as shown in 
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block L of Fig. 25A and as shown in detail in Fig. 26 
by a module SET STAT..' CHK LIST then advances the tail 
object: as shown in block M, and the loop through J, K, 
L and M is repeated. If the tail object status is not 
5 .true, then CHK LIST advances the tail object as shown 
in blopk M, and the loop through J, K and M is 
repeated. 

The module SET STAT as shown in Fig. 2 6 performs 
the sub-steps of comparing the areas of a plurality of 
10 valid objects and designating one of the valid objects 
as a large valid object and the other of the first and 
second valid objects as a small valid object and 
determining whether the small valid object is contained 
in the large valid object as defined by the four 
15 extremum points of the large object for homogeneous 
objects. As shown in decision diamond A of Fig. 26, 
i the first, step of SET STAT is to ask whether the head 
object is larger than the tail object. If so, the head 
object is defined as a large valid object, and the tail 
20 object is defined as the small valid object as. shown in, 
block B. . If the head object is not larger. than the 
tail object, then the head object, is defined as the 
small valid object, and the tail object is defined as 
the large v ; alid object. Decision diamond D of SET STAT 
25 then asks whether the small object is contained within 
the large object. . If .not,. SET STAT stops running as 
shown by oval E. If the small object is contained 
within the large objeqt,. then decision diamond F asks 
whether the edge contrast of the large object is 
3 0 greater than the edg<a contrast .of the small ob.ject, and 
whether the edge contrast of the large object is 
greater than the predetermined minimum edge contrast. 
If the answer to both of these questions is yes, then, 
the large, object status attribute value is set to true, 
35 and the small object status attribute value is set to 
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false as* indicated by' block G, and the module stops 
running as indicated by oval H. If the answer to at 
least one of the questions in decision diamond F is no, 
then the small object status attribute value is set to 
5 true, "the large object status attribute value is set to 
false as indicated by block I, and the module stops 
running as indicated by oval J. 

The method of the' first'/ specific application of 
the second embodiment further includes the step of 

10 ' performing a final" check for redundancies of the valid 
obiect and resolving the redundancies* to prevent 
multiple identification' of the valid object. The final 
redundancy checking step further includes the sub-steps 
of comparing the areas of a plurality* of valid objects 

15 and designating one of the valid objects as a large 
valid object and tlie other of the first and second - 
valid ctiects as a" small valid object and removing the 
large valid object when the small valid object and the 
large valid object overlap. * The final redundancy 

20' checking step is performed by a module, FINAL CHK, as 
illustrated by the' flow chart of Figs. 27A and 27B and 
a module INT STAT, as illustrated by the flow chart of 
Fig. 28. The modules FINAL CHK and INT STAT are the 
* same for both homogeneous * and inhomogeneous objects, 
' - 2 5" and are thus- only illustrated once . - 

r 1 'The first step" of FINAL CHK* is to initialize the 
object attribute value to true for all objects as shown 
in block A of Fig. 27A? The counting index for 
counting valid objects is the initialized to one as 

2 0 ' shown in block bV The head object is defined as the 

initial object in a list of status attribute values as 
illustrated in block C. * Decision diamond D then asks 
whether the counting index is less" than the total 
number of objects. If not, the module FINAL CHK goes 

35 to block A of Fig. 27B. 
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As shown, in block A of Fig. 273, FINAL CHK 
advances to the first object. Decision diamond 3 asks 
whether FINAL CHK is at the last object. If it is not, 
.then decision diamond C asks whether the object status 
5 .attribute value is false. If not, then FINAL CHK 
. advances . to ,the. next . object as shown in block E, and 

decision diamond B again asks whether FINAL CHK is at 
; the last pb ject iV The loop through B, C and E continues 
. until FINAL. CHK is at the next object. If the object 

10 status attribute value is false, then the object is 

deleted as shown in block D . FINAL CHK then advances 
to th,e. next . object as shown in block E, and decision 
diamond B asks whether ; the FINAL CHK at the last 
object. . The loop through B - E' continues until the 
15 . .next object is„ the last object, at which point FINAL 
CHK advances, to. the first object as shown in block F. 
The , count is. then initialized to one as shown in block 
. G. Decision diamond H then asks whether FINAL CHK is 
at the , last , object , If it is not, then the count is 
., 20 incremented as shown in block I, and FINAL CHK advances, 
; ( to thenext object is shown in block J. Decision 

diamond H : again asks whether FIIJAL CHK is the last 
object, and the loop through H, I and J continues until 
; FINAL CHK is at. the last object. Then the total number 
25 of. valid pbjects as contained in count is returned as 
shown by block K of Fig. 2 7B- 

Returning to decision diamond D of Fig. 27A, if 
the counting index is ^ less than the total number of 
objects, then /the. tail object is defined as the next 
30 object beyond the head object .as shown in block E. 

Decision diamond F then asks if the status attribute 
value of the head object is true. If not, the FINAL 
,CHK- advances the head object as shown in block G and 
increments the counting index as shown in block H. 
25 FINAL CHK then returns to decision diamond D, and the 
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loop through d' -' T 'continues until' the status attribute 
value of the head' object is true.' Then decision 
diamond I asks whether the "tail object is not the last 
object and whether the head object status attribute 
value is true. If at least one of these conditions is 
not met, then FINAL CHK advances the head object as 
shown in block G and increments' the' index as shown in 
block H. FINAL CHK then returns" to' decision diamond D, 
and the' loop through D - I continues until the answer 
to both questions in decision diamond I ' is yes. Then 
decision diamond J asks whether tlve tail ; object status 
attribute value is true. If hot, ; FINAL CHK advances 
the tail object as shown ' in block L of Fig. 2 7 A, and 
the loop through I, J and L is repeated- until the tail 
object status attribute value ; is true. Then FINAL CHK 
runs a module INT STAT, as" shown in block" K of Fig. 27A 
and advances the" tail object as shown -in block L. 

The steps of the module TNT STAT as illustrated in 
block K of Fig. 27A are shown in detail in Fig. 28. 
Decision diamond A of Fig. 2 8 asks Whether the 'head 
object is larger than the tail object.' If so, the head 
object Is' defined as the large valid object, and the 
tail object is defined as the small valid object as 
shown in block's. If the' head object is not larger 
"than the tail- object, then the head object is defined 
as the small valid object, and the tail object is 
defined as the large valid object as shown in block C. 
Decision diamond D then asks whether the small valid 
object is contained ih ; 'the large valid object. If not, 
then INT STAT is at " its" end, as shown "by oval E. If 
the small valid object is contained in the large valid 
obiect, then the large object status attribute value is 
set to false as shown in block F, and INT STAT is at 
its end as shown by oval G. 



BNSDOCID: <WO 941©402At_L> 



SUBSTITUTE SHEET 



WO 94/16402 PCT/US93/02736 

60 

Further, in accordance with the present invention, 
there ,is provided a .second, specific application of the 
method of the second embodiment for identifying at 
least one valid object. This application comprises a 
5 method for effectively, removing previously identified 

- , objects from an. image, thus creating a remainder image 

. ..for further searching v The method employs an iterative 
, : methodology, for .identifying new objects in a remainder 
image... The remainder, image is that part of the image 
• 10, .. where- objects have not been identified after a previous 
search of the- image • : This method includes the steps of 
the second, embodiment, ; employing entropic thresholding, 
. and :a .addition, .the., steps shown generally, in the flow 
char-,;vf Figs. 29A and., 29B. ..As shown in block A of 
15. Fie. 2?A, the first step in. 5 the method for removing 

previously identified or valid objects from an image in 
order- to search for. new objects in a remainder image is 

- to. search ..for all previously identified valid objects. 
The next step in the method is to select a 

2 0 predetermined. gray level, values as shown in block B and, 
then to scan the image. to determine all points having 
. the predetermined gray level values as shown in block C 
of Fig . 2 9A. ,.- 

The method of removing previously, identified 
.25 . objects from a ; n image of the present invention further 
includes- the step .of- reassigning the ^ gray level value 
of each ppint having, the predetermined gray level 
value, a, .new predetermined gray level value as shown in 
block D of Fig. 29A. : .The next step of ^the method is to 
30 assign the predetermined gr.ay level value, to all 

interior points of , all previously identified valid 
objects as shown in block E . 

The method of. removing previously identified 
objects from an image of the present invention further 
35 comprises the step of generating a gray. level histogram 
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of a remainder image as shown in block F of Fig. 29A. 
The remainder image comprises all the points having a 
gray' level value other than the predetermined gray 
level value". 

5 ■ " ' The method of remdving previously identified 

objects from an image o£ the present* invention further 
also includes the step of selecting N ; global entropic 
threshold gray levels *a$ shown ih block : G of Fig. 29B 
and subdividing the gray level histogram "'of the 

10 ' remainder image into N"'+ "1 sub-histograms using each of 
the entropically selected threshold gray levels as 
shown' in block H d'f" Fig ! . : : 2 9B . ~ The* method is iterative 
* in that the process" includes - dividing a histogram into 
■ upper and lower histogrkmV for' a predetermined number 

15 of N divisions or iterations so as to create N + 1 sub- 
histograms'. r To Illustrate these steps, Fig. 30 shows a 
gray level histogram having N global entropic threshold 
gray level values; t 1 - t N . The gray level histogram 
of the remainder imafg'e is divided into N ' +' 1 sub- 

20 histograms using each 6f the global entropic threshold 
gray levels! The steps for selecting N global entropic 
threshold gray levels ire performed by - the module 
ENTROPY and are the same as those shown" in the flow 
cha^t : of Fi/g". 13 : V ; ;i ' 

25 The selecting step includes the sub-steps -of 

sequentially pa~rtiti'ohihg the : histogram of the 
reiriain&er image""' at e'ach- gray level into a first and a 
second palrtitibh. :i 'The' eittropy function is then 
computed' for Wa~ch partition.- The entropy function of 

30 the histogram iV defiired as 'the sum of the entropy 

functions of the first and second partitions. A global 
entropic threshold' gray level is then selected such 
that the entropy function of the histogram is 
maximized. The' gray level histogram is then subdivided 

35 using the global entropically selected threshold gray 
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.. level as defined above which .maximizes the entropy 
function of. the histogram as an upper delimiter and a 
lower delimiter to. create an upper histogram and a 
lower histogram. . The partitioning, computing and 

• 5 selecting steps are then .repeated,, where the repetition 
of the selecting step selects a. next successive global 
entropic threshold, gray level. (i The subdividing step is 
then Repeated using the next successiye global entropic 
threshold gray level as . the global entropic threshold 

10 gray level, as defined in , the selecting step to 

iteratively . calculate the N . global entropic threshold 
«, gray, levels. 

The. method ,.9 f removing previously, identified 
objects from an image of the. present invention further 

15 includes r the : step of searching portions of the 

remainder, image corresponding to each sub-histogram 
using each entropically . selected threshold gray level 
value, for at le v ast one new candidate object. If the 
* gray, level value pf the center of mass of the candidate 

.20... . object .is different than , the predetermined gray level, 
then the candidate object is still . considered as a 
candidate object for being a new valid object . If it 
is, not, then the candidate object is Apt a new valid 
...object-.. The candidate; object l>as : at, least one 

23 candidate object . attribute value. The step of 

searching portions of the image is shown , in block I of 
, Fig. 2 9B, . The. searching .step includes the sub-steps of 
scanning the . image , for - at least one candidate object 
using each global entropically selected threshold gray 

30 level ..value and tracing the candidate object having 
boundary gray , level values .determined by each global 
entropically selected . threshold gray level value. The 
st.ep of searching, portions of .the. image is performed by 
the modules SEARCH IMAGE, FIND OBJECT, and TRACE OBJECT 

25 as described above for the first and. second embodiments 
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and as 'shown 'in Figs. 14/ 15 and' 19A and 19B, 
respectively.' In addition', the module CHK GRAY as 
shown in Fig.* 20 may be* r,un for the iterative method in 
order to retain the 'candidate objects which are 

. 5 ' relatively lighter than the background. To identify 
objects darker than the background, the image is 
inverted Immediately a'f ter : it : has been -generated. 

The method of removing 'previously identified 
object's from an image' of ' the "present- invention further 

10 of the prese'tit invention al/s 6 "comprises the step of 
validating the new candidate object having the valid 
object predetermined attribute valufe* for each of the 
sub-histograms, thereby 1 identifying the valid object. 
This step is shown in liiock 'J of Fig . ; 29B .' The 

15 validating step includes the' sub-steps ; of calculating 
the candidate object attribute value and comparing the 
candidate object attribute value to the valid object- 
predetermined attribute 'values to identify valid 
candidate objects. ihe validating : step ^further 

20 Includes the sub-step of checking for redundancies of 
the validate object to prevent multiple! identification 
of 'the valid object. The redundancy checking sub-step 
is performed by the module CHK LIST as described above 
for' the 1 first embodiment 'with respect to Figs. 23A - 
. 2 5 -2 3D, : 25A and 25B, and the module SET STAT as described 
above with respect ' to Figs'. 24 and 26'. 

The validation *- step* of the method of removing 
previously identified 'objects from ah image of the 
present ' invention may comprise the step of filtering 

30 the image: The filtering step for inhomogeneous 

objects is ; performed by the module COL' FILT as shown in 
Fig.. 18 as described above*; 

The method of 1 removing "previously identified 
objects from an image of the present invention further 

35 includes the step of performing a final check for 
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: , redundancies of ^ the valid object and resolving the 

redundancies to prevent multiple identification of the 
valid, object . The_final redundancy checking sub-step 
is performed ,by the ^module FINAL CHK as described above 
..5 for the first embodiment with respect to Figs. 24A - 5 
and the module INT STAT as illustrated in Fig. 25. 

. In .accordance with the first and second 
embodiments of. the. present invention, there is provided 
: - . an image analysis system for identifying at least one 
10. valid pb.ject in a background. The. valid object has at 
le^st one predetermined attribute value which 
represents : the definition of a valid object of an 
; object to. .be identified,. . A block diagram of the system 

, of, , the, present., invention, is shown in Fig. 31. A system 
.15 .for identifying at least one valid pbject in a 
background, is shown generally at 10 in Fig. 31. 

The system of the present invention comprises 
means .for generating an image of the. object and the 
-background. As shown in Fig. 31, the means for 
2 0. generating iniage of the object and the background 
comprises a camera 12... Although a CCD camera is 
generally used with the, present invention, any type of 
camera may bemused without departing from the general 
principles, of the present invention. 
25 The system of. the present invention also comprises 

means for digitizing and. storing the image. The means 
for digitizing,, and. storing . the image comprises a frame 
grabber 14 as shown, in Fig., 31. The frame grabber 
digitizes and stores. the ( video image in one frame, as 
.30 known to one skilled .;Ln the image processing art. 

- Alternatively, the means , for digitizing and storing the 
■ image comprises a video digitizer, which also digitizes 
and stores the. image, although not necessarily in one 
frame.- The system of the present invention further 
35 comprises means for displaying the image. The means 
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for displaving the image "comprises a monitor 16 as 
shown in Fig. 31. 

the svstem of the present invention also comprises 
computer means'. The computer means comprises a 
5 computer system 18 as "shown in Fig.' 31. The computer 
system comprises a central processing' unit (CPU) and a 
memory. The computer means 'also includes a driver 20, 
a kernel 22 and a post-scan filter '26 ; as shown in Fig. 
31. Driver 20 stores the* definition' of' the valid 
10 object. The entropic Kernel 22 generates a gray level 
histogram of the image ahd entropically selects a 
threshold gray level such' that the : entropy^ function of 
the histogram is maximized: Entropic kernel 22 also 
searches" the image fdr at least one candidate object 
15 and validates the 'candidate 7 object hiving the valid 
object predetermined attribute Value to identify the 
valid object. the validated objects'are represented by 
box 24 in Fig. 31 The driver 7 * and the kernel may 
comprise software incorporated ; in- the memory . 
20 Alternatively, the driver a'rid the kernel may be 

programined iht6 a programmable', read-only memory (PROM) 
from' which the software may be retrieved.' The post- 
scan 'filter is shown at 26 in Fig- 31 and provides a 
final check to remove"' redundancies' in overlapping 
23 objects* 1 as described abbve. - - - 

' It will be apparent J 'to those' skilled in the art 
that various modifickti'ons * and variations can be made 
in the methods of "thi* pre'sent invention without 
departing from the' scope or spirit of the invention. 
30 For" example, any type ' of ' entropic thresholding may be 
employed/ such as entropic' thresholding of a co- 
occurrence matrix as disclosed by Pal and Pal, supra. 

The invention will be further clarified by the 
following examples, which are intended to be purely 
3 5 exemplary of the invention. 
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■ In this Example, a c.olony counting system is 
* ; ^ described. Colony counting is a task routinely 

performed in microbiolqgy laboratories for quantifying 
5 the amount , of bacteria in a sample. .The sample can be 
one of a. number of f different types such as blood, food, 
j cosmetics., soil etc. This system identified colonies 
in varying backgrounds and was also able to unclump 
highly, clumped colonies. 
10 ; In addition i to accurate colony counting, it is 

important to.be able to classify colonies by type in 
: cases. where . there is , more . than one species of bacteria 
■ on the same .plate ... . In this Example, both typical and 

■ ... atypical .strains of Salmonella were present. In 
15 addition, background (non-pathogenic) bacteria were 
also; present..- It is also important . to be able to 
v .differentiate, both strains of Salmonella from the 
background : bacteria . In this Example, the agar plate 
was Hektoen - Enteric agar where the typical Salmonella 
20 colonies were black, . the atypical Salmonella colonies * 
were light, green afid the background colonies were 
yellow.. The system of this Example was able to 
classify the colonies by . hue and tiius perform 
. classification of species . on the plate. , 
■ 25 ; i In this Example, an additional complexity was 
presented by the number of colonies in the plate. 
There were 875 colonies in the plate, which would make 
.the visual classification of. the colonies extremely 
tedious . .i The large number, of colonies resulted in 
30 significant, clumping, wtiich. posed another major 

difficulty. The; system described in this. Example was 
able to unclump . the colonies and . then . perform hue 
classification to . classify .the different species on the 
plate . 
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The apparatus used in this Example to count 
colonies comprised a Petri dish holder to hold the 
plate, an optical system for illuminating the plate and 
an image acquisition system to generate an image of the 
5 sample on the plate as described above with respect to 
Via.' '31. A video signal from' a camera was fed into a 
frame grabber', which occupied a slot in a PC. The 
'" v frame' grabber digitized the image. ■■' 

The Petri dish' holder comprised a movable, 
10 : machined member'' which' could accept standard Petri 

dishes . The member haft- a countersunk "circular indent 
"capable of" accepting the dishes and was slid into place 
• ' via a bearing-based support ! . '' Below the holder, there 

was 1 ' room "to insert ' ah ' appropriate background sheet to 
15 enhance image contrast . > - 

; The illumination source used : comprised a Philips 
FC 8T9/CW circular fluorescent lamp (8" (20.3 cm.) - 
diameter) which surrounded the Petri dish . The lamp 
was' mounted approximately 1" • (2 .'54 cm. ) above the dish 
20 'and illuminated the dish uniformly from all sides. The 
distance between the lamp and the dish was important. 
If the lamp was too high, the resulting image would 
have revealed significant "glaring" and "haloing" from 
the curved surfaces of the colonies. The colonies can 
25 "beacon side red- small 3D lenses, and oblique, grazing 
angle illumination was' necessary so that light 
scatter'ing : from the lens-like surfaces did not enter 
the camera. : The lamp ; was actively stabilized using a 
Mercrbn FL0416-2 controller, which kept the light level 
30 ' constant, to prevent fluctuations in output. In 

addition, the controller was ■ remotely programmable to 
adjust the brightness of the 'lamp. This feature was 
especially useful for colony plates , where the inherent 
contrast was variable. For low-contrast plates, the 
35 light level can be increased to improve the contrast. 
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»' The optical system, comprised a Sony -XC-7 7 
monochrome CCD video .camera with; a 16 nun. F2 . 9 Nikkon 
* *■ lens.- The camera* was centered above the Petri dish at 
a distance of approximately 1.5 feec (0.05 m.). The 
5 distance from the camera to the dish was - such that the 
disk 5 just filled the surface of .the, CCD sensor in order 
to maximize spatial resolution. 

A filter wheel was placed between,* the lens and the 
camera 'sens'orv- The filter wheel had four positions. 
10 The filter combination in . each position are described 
below: 

.... Table I 
Position Filter COMBINATION 

1 1 (BW) " " ** l.£ ND filter (ND=neutral density) 

15 • 2 ' (RED) Kodak filter #25. (catalog #149 5605) 

+ 0.5 ND filter 

3 (GREEN) Kodak filter #58 (cat* 149 5860) 

+ 0.3 ND filter 

4 (BLUE) Kodak filter #47B (Cat. #149 5795) 

2 0 . ' -• , - 

A black light shield made of a crystalline homopolymer 
of formaldehyde, sold under the trademark "DELRIN" by 
E . I. du Pont de Nemours and Company of .■ Wilmington, 
Delware, extended from the camera to the plate to 

2 5 prevent stray light from entering the camera. The 

video signal from the camera went directly to the frame 
grabber on the PC. 

The image acquisition system comprised a Data 
Translation DT2851 frame grabber board which occupied 

2 0 an expansion slot on the PC. -A Dell 33 MHZ 386AT 

system was used to store images and execute the method 
of the present invention, using color as an attribute 
value. The frame grabber board had a 512 x 512 pixel 
video buffer in which the digitized image was stored. 

35 In order to minimize "shot" noise from the camera, 
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several image frames- were averaged, and the resulting 
1 averaged image was stored, and. used for. further 
analysis.- The output of the frame grabber was 
connected to a" Sony TRINITRON monitor to display the 
5 • image . . i • -, 

The following (1) ---(4) were computer 
specifications : - ; . - 
: (-1) ■ Driver*^ ■ . : . 

The attributes and their; ranges- for valid 
10 colonies 1 are ishown- in. Table- 2 below. v 

Table 2 

Ranye of Attribute values for screening 

Note :: * -Only some of the attribute values required valid ranges. 
15, r t o. Others were "descriptors" which did .not, influence object 
- ' - .validity. 



Attribute 



Valid Range 



20 (1) xcm — x coordinate of object 
* . r 1 center of mass . 



DESCRIPTOR 



(2) ycm « y coordinate of object 

:' center of mass .i 

(3) [ perim - total length of perimeter 

of object 



DESCRIPTOR 



DESCRIPTOR 



30 



(4) npts - number* of perixreter points 
* « in object npts > 1 3 r .. \ • > 



(5) obj_type - type of object 
- (Exterior or Interior) 



DESCRIPTOR 
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Exterior = object never contained within 
another object 

Interior » object previously contained 
i , within, another object 



(6) area « area of object 



area > 0.0 



, . (7) shape - shape factor of object 
10 (8) status - status value of object 

which is used in artifact removal 
and redundancy checking in several 
different ways .- 



shape > - 0 . 8 
DESCRIPTOR 



15 (9) contrast -» edge contrast, of object 



contrast > 100 



20- 



(10) edge * data structure containing 

r coordinates of .extremum points 

; i ;Of Object 

(11) thresh -/threshold at which object 
. r was. detected 



DESCRIPTOR 



DESCRIPTOR 



(12) max — maximum gray, level of histogram. - - . * DESCRIPTOR 
2 5 partition in which object was detected 



(13) clump_ shape « maximum shape factor for a 
clump of candidate objects 



UNUSED 



30 (14) clump_size » minimum area for a, clump, of 
. candidate objects . 



UNUSED 



(15) . worst_shape - minimum shape factor for a . . UNUSED 
clump of candidate objects 
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(16)' hue mean - mean "hue of object- 



huemean > 1.90 
and huemean <3.40 
(if huemag >200) 



5 (17) hue mag 



» magnitude of hue vector of object 



10 



(18) hue cm - hue vedtor ot cm 



subject to 

classification 

Rules 

DESCRIPTOR 



15 



(2) Artifact. Removal: 

The following rules were used to eliminate 



ar 



~ ** " * j» ~ 



acts in the agar: 



if the mean hue of a candidate colony was 
less than 1 . 90 and the hue magnitude was greater than 
200, the candidate colony was deleted. 

• If the mean hue of a -candidate colony was 
20 less than 1.90 and the hue magnitude was greater than 

' 200, the candidate colony was deleted. 

• If the mean hu£ of a candidate colony was 
greater than 3.40 and the hue magnitude was greater 
than 2 00, the candidate colony was deleted. 

2 5 " (3) "Redundancy check*:- - - 

The redundancy check for inhomogeneous objects 
described above with respect tO J Figs. 23A: - 23D and 
Fig. 2 4 was used for 1 * -redundancy-checking in this 

Example . 
2 0 (4) Colony Classif i'cation : 

The rules used to classify the colonies into 
the three categories (Atypical Salmonella, Typical 
Salmonella and Background) are summarized below : 
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• If the hue magnitude was less than or equal 
to 200, the colony was classified as TYPICAL 
Salmonella . 

• If the mean hue was greater than 1.90 and 
5 was , less than 2.55, the colony was classified as a 

BACKGROUND colony. 

• If the mean hue was greater than or equal to 
.2.55 and was less than 2.70: 

• If the hue magnitude was less than 2000 or 
10. the hue, of center of mass of the colony plus a hue 

offset of 0.05 is .less than the mean hue, the colony 
; was classified. as a BACKGROUND colony. ' 
~. . .„ • Otherwise, the colony was classified as an 

. . ATYPICAL colony. . ' 

L5 : . . , Otherwise all other valid colonies were 

.. classified as .ATYPICAL colonies . 

A total of 875 valid colonies were identified. 
. Classification of the colonies by hue resulted in the 
20 following breakdown: * 
Typical Salmonella : 155 
; Atypical Salmonella : 385 
Background : 335 
These . results agree very well with the results expected 
-2 5 . from a. visual, examination of the plate. 

Other embodiments pf the invention will be 
apparent to those skilled in the art from consideration 
-of the -specification and practice of the invention 
30 disclosed herein. It is intended that the 

. specification and examples be considered as exemplary 
.only, witiv a true scope and spirit of the invention 
: being, indicated by the^, following claims. 
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WHAT IS P T.ATMED' IS: 

1 . A method for determining the interior points 
of . an object in a background, comprising the steps of: 
5 (a) generating an image of the object and the 

background; 

(b) searching the image for the object; 

(c) determining the perimeter points of the 

object; 

10 (d) creating a perimeter buffer, the 

perimeter buff er 'comprising the perimeter points, each 
perimeter 'point having a row position coordinate value, 
a column position coordinate value and a direction code 
vector, the direction code vector comprising a first 

15 and a second direction dode value for each perimeter 
point, the first direction code value describing the 
relationship of each respective perimeter point to the 
next perimeter point and the second direction code 
value describing the relationship of each respective 

20 perimeter point to the previous perimeter point; 

(e) sorting the perimeter points in the 
perimeter buffer in a predetermined order to create a 
sorted perimeter buffer; 

(f) assigning to each perimeter point in the 
25 sorted perimeter buffer a status variable + having one of 

a first and a second value, the value of the status 
"variable being determined by the first and second 
direction code values; and 

(g) "designating as the interior points of the 
30 object each p6irit on the same line segment as the 

perimeter point being "examined and lying between the 
examined perimeter point arid "the next perimeter point 
in the sorted' perimeter buffer, the examined perimeter 
point being assigned the first value of the status 
35 variable. 
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2. The method of Claim, 1, wherein the step of 
searching the image comprises .searching <?nly a portion 
of the object. 

3. The method. of Claim 1, wherein the step of 
5 searching the image comprises tracing the object 

sequentially . in the clockwise direction. 

,4. The method of Claim 1 , wherein, the step of 
determining . the perimeter points of the object 
comprises tracing the 7 object sequentially .in the 

10 counterclockwise direction. 

5. . The method o.f Claim 3, wherein the sorting 
step comprises the sub-step of sorting the perimeter 
points in the perimeter buffer by row. in a 
predetermined .order to create a. row-sorted, perimeter 

15 , buffer h comprising, at least one row-sorted perimeter 
point. 

6- The method of Claim 5, wherein the sorting 
step comprises the additional sub-step of sorting the 
perimeter points in the row-sorted perimeter buffer by 
20 column in a . further. predetermined order to create a 

row-column sorted perimeter buffer comprising at least 
one row-column sorted perimeter point . 

.7. The., method of Claim .6, wherein, the sub-step 
of sorting the perimeter ..points by row comprises 
,25. sorting the perimeter points in ascending order, such 

that the points with , the lowest row-position coordinate 
.values f are placed at { the . .start of the buffer. 

8.. The. method^of Claim. .6, wherein -the sub-step 
of sorting the perimeter points by row, comprises 
30 sorting the perimeter points in descending order such 
that the .points : .wi.th. the highest row-position 
coordinate val,ues are placed . at. the start of the 
buffer . ...... 

9- The method of Claim 7, wherein the additional 
35 sub-step of sorting the perimeter points by column 
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comprises sorting the row-sorted perimeter points in 
ascending order such that the point with the lowest 
row-position coordinate value and the lowest column- 
position coordinate value is placed at the start of the 
"3 buffer". 

10. The method of Claim 8, wherein the additional 
"sub-step of sorting the perimeter points by column 
comprises sorting the "row-sorted perimeter ' points in 
ascending order such that the point ' with the highest 
10 row-position coordinate value ' and the lowest column- 
position coordinate value is placed at the start of the 
' ' buffer. 

" 11. The method of Claim 3, wherein the sorting 
• * step comprises the' sub-stej? of sorting the perimeter 
15- points* in "the perimeter buffer by 'column in a 

predetermined order to create a column-sorted perimeter 
buffer comprising ait least on4 column-sorted perimeter 
point ."''*" ' - 

"12. The method of Claim 11, wherein the sorting 
20 step comprises the additional sub-step of sorting the 
perimeter points in the column-sorted' perimeter buffer 
by row in' a further predetermined order' to create a 
columns-row sorted perimeter buffer comprising at least 
one column-row sorted perimeter point.' 
'25 - '13 . The method' of Claim 12, wherein the sub-step 
of s'orting the perimeter points by column comprises 
sorting the perimeter joints in descending order such 
^ that the point's with the highest column-position 
coordinate Values ' are 'placed at' 'the' start of the 
' 3 0- ' "buffer. ' ' r ■ ' 

14. The method of Claim 12, wherein the sub-step 
of sorting the" perimeter points by column comprises 
sorting the perimeter points in ascending order such 
that the points with the lowest column-position 
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; coordinate values are placed .at the start of the 
buffer. ... 

.15... The method, of Claim 13, wherein the 
additional sub-step of sorting the perimeter points by 
5 . row comprises sorting the column-sorted perimeter 

points ia descending order such that the i point with the 
- highest row-position coordinate value and the highest 
column-position coordinate . value is placed at the start 
of the buffer. : .... 

■ 10 . 16 ; The method, of Claim 14, wherein the 

additional sub-step of sorting the perimeter points by 
row comprises .sorting, the column-sorted perimeter 
points .in descending order such that the point with the 
lowest column-position coorclinate value and the highest 
15 .row-position coordinate value is placed at the start of 
. the .buffer. ... 

'17., , The method of Claim 4, wherein the sorting 
step comprises the sub-step of sorting the perimeter 
points in the perimeter buffer by row in a 
• 2.0-. predetermined order to create a row-sorted perimeter 
buff er.- comprising at least one row-sorted perimeter 
. point : , • 

18. The method of Claim 17, wherein the sorting 
step comprises the additional sub-step of sorting the 

25 • .perimeter points in the , row-sorted perimeter buffer by 
column in a further predetermined order. to create a 
t row-column sorted perimeter buffer comprising at least 
one row-column sorted perimeter point. 

19. : The method of Claim 18, wherein the sub-step 
20 of sorting the perimeter points by row comprises 

sorting the perimeter points, in .ascending order such 
that the. points with the lowest row-position coordinate 
values are placed at the start, of the buffer. 

20. The method of Claim 18, wherein the sub-step 
3 5 of sorting the perimeter points by row comprises 
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sorting the perimeter points • in descending order such 
that the points with the highest row-position 
coordinate values are placed at the start of the 
buffer. 

' : 2i. The 'method of' Claim 19-, wherein the 
additional sub-step of sorting the perimeter points by 
column 'comprises sorting the row-sorted perimeter 
points' in descending order such' that ■ the point with the 
lowest row-position coordinate value and the highest 
column-position coordinate value is placed at the start 

"of the 'buffer. 

"22. The method of Claim 2D, wherein the 
additional sub-step Of Porting the; perimeter points by 
column comprises sorting the. rbw-'sorted perimeter 
15 ' points in- descending 1 order such that the point with the 
highest row-position coordinate value and the highest 
column-position coordinate value is placed at the start 

of the "buffer. - 

23/ The method of Claim wherein the sorting 

20 step comprises the sub-step of --sorting the perimeter 
points in the' perimeter buffer by column in a 
predetermined order to create a column-sorted perimeter 
buffer' comprising at least one column-sorted perimeter 
point. 

25 2 ! 4. The method of Claim 23, wherein the sorting 

step comprises the addition SI sub-step of sorting the 
perimeter points in the column-sorted perimeter buffer 
by row in a further predetermined order to create a 
" column-row ; sorted perimeter buffer comprising at least 
30 one column-row sorted" "peximeter point. 

25. : The method of Claim 24, wherein- the sub-step 
of sorting the' perimeter points by column comprises 
■ • 'sorting the perimeter points in descending order such 
that the points with the highest column-position 
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coordinate . values are placed, at. the .start of the 
buffer. 

,26. The method of Claim 24, wherein the sub-step 
of sorting the. perimeter points by column comprises 
5 sorting the perimeter points in ascending, order such 
that the points .with the. lowest column-position 
coordinate values, are placed at the start of the 
■ . ( buffer, 

27. The method of Claim 25, wherein the 

10, additional sub-step^of sorting the perimeter points by 
ro.w comprises sorting ..the column-sorted perimeter 
points in ascending order ,such that the point with the 
, lowest row-position, coordinate value and .the highest 
r column-position coordinate value is .placed at the start 
15 : of the buffer. 

28. The method of Claim 2 6, wherein the 

- additional sub-step of sorting the perimeter points by 
row comprises sorting the column-sorted perimeter 
points in ascending order, such that the point with the 
lowest column-position coordinate value and the lowest * 
row-position coordinate value is placed at the start of 
the buffer.. 

.29.^ The method of Claim 6, wherein the 
designating step comprises the sub-step of designating 
25 as the interior points of the pbject each point on the 
same row as the perimeter point being examined and 
lying between the examined perimeter point and the next 
perimeter point in the row-column, sorted perimeter 
buffer. 

30 : , 30.. The method (i of Claim. 12, wherein the 

designating step comprises the sub-step of designating 
as the. interior points of the object each point on the 
same row ; as the perimeter point .being, examined and 
lying between, the examined perimeter point and the next 
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perimeter point in the column-row sorted perimeter 
buffer. 

31. The 'method" of Claim i8, wherein the 
! designating step" comprises the sub-step of designating 
5 'as th4 interior 'points " of the object "each point on the 
same column as the p'erirtieter ' point being examined and 
lying between tlie' examined perimeter point and the next 
perimeter point in the row-column sorted perimeter 
buffer'. 

' 1 10 " ' 32 . The method of Claim 24, wherein the' 

designating step comprises the sub-step "of designating 
ks the interior 'points of the object each point on the 
sam£ column as the perimeter 'point being examined and 
iyirig' -between the exairiihed pe'riirieter 1 point and the next 
15 perimeter point in the column-row sorted perimeter 
buf f er "t 

* ! 3'3. The method of Claim 1, further Including the 
step's of : 

(h) determining the gray level value of each 

■ 20 interior point, and 

(i) characterizing the object by using the 
gray level values of the interior points. 

34. The method of Claim 33, wherein the 

' * characterizing step cdmprises determining the texture 
' " 25~ ' of 'the object. 

35. The' method of : Claim 33, wherein the 
characterizing "step comprises determining the color of 

the object 

36. The method of Claim 33 p wherein the 

30 characterizing step comprises determining the patterns 
in the bbje'ctv 

" 37 : :' the method of Claim 1, further comprising the 
step of labeling the ; object by assigning 'a 
predetermined gray level value to each interior point 

35 designated in step (g) - 
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...38. The . method of Claim 1, further comprising the 
step, of checking, for each redundant perimeter point. 

39. The. method of Claim ,38, further comprising 
the step of designating all points lying between each 
redundant perimeter point and the next perimeter point 
on the. line, segment as the interior points of the 
object only when all instances of each redundant 
perimeter pp,int ..are assigned the first value of the 
status variable. 

4 0,. The method of Claim 1, wherein the step of 
searching , the .image includes the sub-steps of: 

. c (i) . generating a gray level histogram of 
the image .the. gray, .level histogram haying an entropy 
function, 

i . . (ii) entropically selecting a threshold gray 
.level value .such that the entropy function of the 
histogram is. maximized, 

(iii) , scanning, the image- for at least one 
object using t the. entropically selected threshold gray 
level value, and .* 

(iv) tracingr the candidate object having 
boundary gray level values determined by the 
entropically selected threshold gray level value. 

. 41 . A method of ^identifying at least one valid 
object having, at least one valid object predetermined 
attribute value in a background, comprising the steps 

of: ... ; « f • * r . • 

. . (a) generating, an. image of the object and the 
background; 

. (b) ^searching the image for at least one 
candidate object using at least one automatically 
calculated threshold, wherein the candidate object has 
at. least one candidate object attribute value; 

(c) determining the perimeter points of the 
candidate object; 



SUBSTITUTE SHEET 



WO 94/16402 FCt/US93/02736 

81 

(d) creating "a perimeter buffer, the 
perimeter buffer comprising the* perimeter points, each 
perimeter point having a row-position coordinate value, 
a column-position coordinate value arid ^"'direction code 

5 vector, the direction code' vector comprising a first 
and a second direction code value for each perimeter 
point/ the first direction code value ; describing the 
relationship of each respective 'perirheter point to the 
next perimeter point and th£ second direction code 
10 value describing' the' relationship of each respective 
perimeter point to the previous perimeter point; 

(e) sbrtifig this perimeter points in the 
perimeter buffer in at predetermined order to create a 
sorted perimeter buffer; 

15 ' ' "(f) assigning 'to' each perimeter point a 

status variable having one of a first and a second 
value a first and a second status Variable, the value 
of the status variable bVing determined by the first 
and second direction code values; 

20 (g) designating as the interior' points of the 

object each point on the same line segment as the 
perimeter point being examined and lying between the 
examined perimeter point and the next perimeter point 
in 'the sorted perimeter buffer, the Examined perimeter 

2 5 point "being assigned the first value ofvthe status 
variable; 

(h) characterizing the candidate object by 
"* the candidate object' 1 -'attribute value; and 

(i) validating the candidate object having 
30 -the valid object predetermined attribute value to 

identify the valid object'. 

42. The method of ' Claim 41, wherein the step of 
searching the image comprises searching only a portion 
of the object. 
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■43- The method of Claim 41, wherein the step of 
, searching the. image includes .the sub-steps of: 

(i) generating a gray level histogram of 
the image, the gray level histogram having an entropy 
5 function, 

(ii) .entropically selecting a threshold gray 
level value such that the entropy function of the 
histogram is maximized, 

(iii) .scanning the image for at least one 
10 t object using the : entropically selected threshold gray 
level value, and 

::. (iv) tracing the candidate object having 

boundary ..gray levels determined by .the entropically 
. selected threshold gray level value. 
15 t - t 44.- t The method of Claim 41, wherein the step of 

determining the perimeter points of the object 
^ comprises tracing the . object sequentially in the 
clockwise direction. 

,< .45. The method of .Claim 41, wherein the step of 
20 determining the perimeter points of the object 

comprises tracing the object sequentially in the 
counterclockwise direction. 

46. The method of Claim 44, wherein the sorting 
step comprises the sub-step of sorting the perimeter 
25 points in the perimeter buffer by row. in , a 

\ . predetermined; order - to ' create a ; row^sorted perimeter 
buffer comprising at least .one row-sorted perimeter 
point . • ,. .. 

47., The method of Claim. 46, wherein the sorting 
step comprises the additional sub-step of sorting the 
perimeter points in the row-sorted perimeter buffer by 
column, in a further predetermined order to create a 
row-column sorted perimeter buffer comprising at least 
one row-column sorted perimeter point. 
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' 48. The" method" 'of Claim" 47, wherein the sub-seep 
of sorting the perimeter "points by row comprises 
sorting the perimeter points in ascending order such 
that the' points with the lowest row-position coorainate 
z values are placed at the start of the buffer. 

' l 49. ' The 'method of Claim 47, wherein the sub-step 
of sorting the' perimeter" points by row comprises 
scrting the perimeter -points' in 'descending order such 
that the points with the highest row-position 
10 coordinate values are placed at ; the start of the 
buffer. 

' 50. The method of Claim 48, wherein the 
additional sub-step of sorting the perimeter points by 
coiunr. comprises sorting the ; row-sorted perimeter 
15 poir.ts 'ir. ascending order such that the point- with the 
lowest row-position coordinate value and the lowest 
column-position coordinate value is placed at the start 

of the buffer. 

5l. The method of Claim 49, wherein the 
20 additional sub-step of sorting the perimeter points by 
' column comprises sorting the row-sorted perimeter 

points in ascending order such that the point with the 
: highest row-position coordinate value and the lowest 
cbiumn^position coordinate value is placed at the start 

25 of' the buffer. '" ' 

" - • •52-. The method-of Claims 44, wherein the scrting 
step comprises the -sub-step of sorting the perimeter 
points in the perimeter buffer by column in a 
predetermined order to' create a column-sorted perimeter 

30 buffer" comprising at least ohe : column-sorted perimeter 
point . 

" 53. The 'method of : Claim 52, wherein the sorting 
step comprises the additional sub-step of sorting the 
perimeter points in the column-sorted perimeter buffer 
35 by row ir. a further predetermined order to create a 
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column-row sorted perimeter, buffer comprising at least 
one column-row, sorted, perimeter point. 

54. The method of , Claim 53, wherein the sub-step 
of sorting the . periineter points by column comprises 
. 5 sorting the perimeter points in descending order such 
that the points..with the highest column-position 
coordinate values are placed at the start of the 
■ - buffer . 

55 ; . ( The method of. Claim 53, wherein the sub-step 
10 ; of sorting the perimeter points by column comprises 
^sorting the perimeter points in ascending order such 
that .the points with ( , the .lowest, column-position 
. >.: coordinate values ar;e placed at the start of the 

buffer . •• . r i. 
15 5 6 . -■ The. method of Claim 54, wherein the 

■additional sub-step of sorting, the. perimeter points by 
row comprises sorting the column-sorted perimeter 
points in descending order such, that the point with the 
highest row-position coordinate value and the highest 
20: c.plumn-position coordinate value is placed at the start 
_ of the buffer,. 

.57. The method of Claim 55, wherein the 
additional sub-step of sorting the perimeter points by 
row comprises sorting the column-sorted perimeter 
25 ... points in descending order such that . the point with the 
lowest column-position coordinate value, ^nd the highest 
row-position coprdifiate., yalu.e . is placed . at the start of 
the buffer. , r . „ . 

58. The method , of .Claim 45,. wherein the sorting 
30 step comprises the.subrstep f of sorting the perimeter 
points in the perimeter buffer by row in a 
predetermined order to create a row-sorted perimeter 
buffer comprising at least one row-sorted perimeter 
point . 
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59. The method of Claim 58 , - wherein- the sorting 
step comprises' the additional sub-step of sorting the 
perimeter points in the' row-sorted perimeter buffer by 
"column" in a further predetermined order to create a 
5 row-column sorted perimeter buffer comprising at least 
one row-column sorted perimeter point. 

'60. The method of- Claim 59/ wherein the sub-step 
of sorting the perimeter points by row comprises 
sorting the : perimeter point's in ascending order such 
10 that the points with- the lowest row-position coordinate 
' values are placed at the start of the- buffer. 

'61 1 The method of 5 Claim -59, wherein the sub-step 
of sorting the perimeter points by-row comprises 
sorting the perimeter points in descending order such 
15 "that the points with the highest row-position' 

coordinate values are -placed at the - start of the , 
buffer. 

62. The method 'of Claim 60, wherein the 
additional sub^-step of sorting "the perimeter points by 
20 " column Comprises sorting the' r6w-sorted- r . perimeter. 

points in descending order such that the point with the 
loWest row-position coordinate value and the highest 
column-position coordinate value is placed at the start 
of the^ buffer. "' ' '"" 

- 25 ' v " : 63." The method of Claim 61, wherein the 

additional sub-step of sorting the perimeter points by 
* ' c6liimh"compriseS-iorting j - the row-sorted perimeter 

points in descending order such that the point with the 
highest row-position coordinate value and the highest 
30 column-positibn'-cobrdihate- value is placed at -the start 

of the buffer. = " J "'- ' " ' - 

64. The method ; of Claim 45, wherein the sorting 
step comprises the sub-step of sorting the perimeter 
points in the perimeter buffer by column in a 
35 predetermined order to create a column-sorted perimeter 
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buffer comprising at least one column-sorted perimeter 
point . 

65. The method of Claim 6'4 , wherein the sorting 
step comprises the additional sub-step of sorting tne 

5 perimeter points in* the column-sorted perimeter buffer 
by row in a further predetermined' order to create a 
column-row sorted perimeter buffer comprising at least 
one column-row sorted perimeter point. 

66. The method of Claim 65, wherein the sub-step 
10 of sorting the perimeter points by column comprises 

sorting the perimeter points in descending order such 
that the points with the highest column-position 
, ., . . coordinate values are placed at the start of the 
' buffer " " ' " 

15 . . The method of Claim 65, wherein the sub-step 

of scrtinj the perimeter points by column comprises 
sort'::.? t.^.e perimeter points in ascending order such 
that tr.e points with the lowest column-position 
coorciir.ate values are placed at the start of the 

20 buffer. ' ' '" ' ' ' 

68. The method of Claim 66, wherein the 
additional sub-step of sorting the perimeter points by 
row comprises sorting the column-sorted perimeter 
points m ascending order such that the point with the 
lowest row-position coordinate value and the highest 
column-position coordinate value is placed at the start 
of the buffer. 

69. The method of Claim 67, wherein the 
additional sub-step of sorting the perimeter points by 
row comprises sorting the column-sorted perimeter 
points in ascending order such that the point with the 
lowest column-position coordinate value and the lowest 
row-position coordinate value is placed at the start of 
the buffer . 
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70. The method of Claim 48, wherein the 
designating step comprises the sub-step of designating 
as the interior points of the object each point on the 
same .row as the perimeter point being examined and 

5 lying between the examined perimeter point and the next 
perimeter point in the row-column sorted perimeter 
buffer. 

71. The method of Claim 53, wherein the 
designating step comprises the sub-step of designating 

10 as. the interior points "of the object each point on the 
same row as the perimeter point being examined and 
lying between the examined perimeter point and the next 
perimeter point in the column-row sorted perimeter 
buffer . 

15 72. The method of Claim 59, wherein the 

designating step comprises the sub-step of designating 
as the interior points of the object each point on the 
same column as the perimeter point being examined and 
lying between the examined perimeter point and the next 

20 perimeter point in the row-column sorted perimeter 
buffer . 

73. The method of Claim 65, wherein the 
designating step comprises the sub-step of designating 
as the interior points of the object each point on the 
25 same column as the perimeter point being examined" and 

lying between the examined perimeter point and the next 
perimeter point in the column-row sorted perimeter 
buffer. 

74 1 The' method of Claim 43, further comprising 
30 the steps of: 

( j) subdividing the gray level histogram into 
an upper histogram and a, lower histogram using the 
entropic threshold gray level value as defined by step 
(b) (ii) as an upper delimiter and a lower delimiter, 
35 and 
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( t k) recursively, repeating steps (b) (ii) and 
(b) (iii) and .step. . (b) <.i) for each of the upper and 
lower histograms, wherein the repetition of step 
.(b) (ii) selects a next successive entropic threshold 
5 aray level value, ...thereby .recursively . partitioning the 
gray level histogram, until . a predetermined minimum 
number of new valid qbjects is identified. 

75. The method of Claim 74,, further including the 
steps of : . ; - , ^ : i - 

.. 10- . (1) determining .the ( gray level value of each 

interior point, and 

I , ■ , v ,, :(m) : ..characterizing the object by using the 
gray -level values of the interior points, after 
designating step; (g) .-. ?. *: ■• . 

15 : 76. The method ( of. .Claim 75, wherein the 

f . characterizing , step comprises determining the texture 
.of ,the object „ . c 

,77.. The method of Claim 75, wherein the 
.characterizing .step comprises determining the color of 
2 0 the object . .-. 

78. The method of Claim 75, wherein the 
characterizing step comprises determining the patterns 
in the object. :. 

79. The method of Claim 75, further comprising 
25 ; the step of labeling, the object by assigning a 

predetermined gray .level value, to each .interior point 
designated in sJbep. : (g) ; . . - - 

80. The method ,of - Claim 41, further comprising 
the steps of: - .■ ' 

30 . (j) selecting a predetermined gray level 

value, ;.,-!. v ■ - • --~ 

(k) scanning the image to determine all 
points having the predetermined gray level value, 
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(1)' re-assigning the gray level value of each 
point having the predetermined gray level value a new 
predetermined gray level value, 

0 (m) assigning the predetermined gray level 

5 value to all interior' point's of all valid objects, 

(n) ' generating a gray -level histogram of a 
remainder image, the remainder image comprising all the 
points having a gray level value other than the 
predetermined gray level value, : 1 

10 (n) ' selecting N global entropic threshold 

level values, - 

(o) subdividing the gray level histogram into 
N + 1 sub-histograms using each of the eritropically 
selected threshold gray level values, 1 - 
15> ! (p)' searching portions of the image 

corresponding to each sub-histogram using each global 
entropically selected threshold gray 'level value for at 
least one new candidate object, wherein the new 
candidate object has at least one candidate object 
20 attribute value, and - ■ 

(q) validating the new candidate objects 
• - having the valid object predetermined attribute values 
for each of the sub-histograms to identify the valid 
object . 

25 -81./ The method of Claim 80, further including the 

step of determining whether the gray level value of the 
center of mass of the candidate object is different 
than : the predetermined" gray J level value. 

82. The method of Claim 81, further. including the 
30 step 'of applying a post-scan filter based on the 

attributes of the valid objects validated in step (i). 

83. The* method' of Claim 81, further including the 
step of merging the list of non-redundant, valid, newly 
identified objects with the valid object validated in 

35 step (i) . 
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